I have a function that saves some text to a file:
(defn save-keypair
"saves keypair to ~/.ssb-clj/secret"
[pair file-path]
(let [public-key-string (->> (:public pair) (.array) (byte-array) (b64/encode) (bs/to-string))
secret-key-string (->> (:secret pair) (.array) (byte-array) (b64/encode) (bs/to-string))]
(spit file-path (str "Public Key: " public-key-string))
(spit file-path (str "\nPrivate Key: " secret-key-string) :append true)))
It works fine (currently checking via just opening the file and looking at it myself). However, I'd like to write an actual test to check that everything is working correctly. Is there an idiomatic way of doing this in Clojure?
Look into using with-redefs, as part of your unit tests. In your case, you probably want to merge the writing of the public and private keys into a single form which we can exploit for the test:
;; compute public-key-string and private-key-string as before
(let [contents (format "Public Key: %s\nPrivate Key: %s"
public-key-string secret-key-string)]
(spit file-path contents)
A test could be something like:
(deftest saving-keypair
(testing "Successful save"
(let [file-mock (atom nil)]
;; During this test we redefine `spit` to save into the atom defined above
(with-redefs [spit (fn [path data] (reset! file-mock {:path path :data data}))]
;; Perform IO action
(save-keypair "~/.ssb-clj/secret" {:public "XXXX" :private "YYYYY"})
;; Test if the expected data was saved in the file-mock
(is (= {:path "~/.ssb-clj/secret" :data "Public key: XXXYYYZZZ\nXXXYYYZZ"}
#file-mock))
Use Java interop with File
https://docs.oracle.com/en/java/javase/11/docs/api/java.base/java/io/File.html
In particular, see File.createTempFile() and either file.delete() or file.deleteOnExit(). So you create a temp file, use that in your unit test, reading the file that you just wrote and verifying contents. Then either delete the file explicitly (ideally inside of try/finally) with auto-delete as a backup.
Depending on how you set up the expected results in your tests, you may find the following useful:
clojure.string/trim
tupelo.string/collapse-whitespace
tupelo.string/nonblank=
These helper functions are especially useful for text file output, where the presence of a trailing newline char can be OS dependent. They are also helpful to ignore differences due to the "newline" being CR, CR/LF, or LF.
Related
I'd like to make a unit-test like framework that allows me to write custom tests for individual rules. I'd like each test to be in it's own file, i.e. test_R1.clp would be the test file for rule R1. Each test should be able to load it's own facts file. I've tried many variations of the following, including using a different defmodule for each file. Is what I'm trying to do even possible in CLIPS? If so, what else is needed to make this work?
I'd like to run my tests via:
$CLIPSDOS64.exe -f2 .\test_all.clp
With the current example, the error I get is
[EXPRNPSR3] Missing function declaration for setup-tests.
I've gotten a single test to work correctly using a unique defmodule for each file (i.e. UNITTEST for the testing framework and R1 for the test_R1 file). However, I would still get errors because of the automatic switching between focus statements when files are loaded, or when functions are defined in other files. I've looked at the basic and advanced CLIPS programming guides, but if I've missed something there, please let me know.
Other specific questions:
Since some tests may load facts that overwrite existing facts, how do I prevent getting errors from redefining existing facts? Do I need to do a (clear) in between running each test?
TestingFramework.clp:
;;; File: TestingFramework.clp
(defglobal ?*tests-counter* = 0)
(defglobal ?*all-tests-passed* = TRUE)
(defglobal ?*failed-tests-counter* = 0)
(deftemplate test_to_run
(slot testid)
(slot testname)
(slot testsetupfunc)
(slot testcheckfunc))
(deffunction test-check (?test-name ?test-condition)
(if (eval ?test-condition)
then (printout t "SUCCESS: Test " ?test-name crlf)
(printout test_results_file "SUCCESS: Test " ?test-name crlf)
(return TRUE)
else (printout t "FAILURE: Test " ?test-name crlf)
(printout test_results_file "FAILURE: Test " ?test-name crlf)
(return FALSE)))
(deffunction setup_tests ()
(open "test_summary_results.txt" test_results_file "w"))
(deffunction finish_tests ()
(close test_results_file))
(deffunction add_test (?test-name ?test-setup-func ?test-check-func)
(bind ?*tests-counter* (+ 1 ?*tests-counter*))
(assert (test_to_run (testid ?*tests-counter*)
(testname ?test-name)
(testsetupfunc ?test-setup-func)
(testcheckfunc ?test-check-func))))
(deffunction run_all_tests ()
(printout t "About to run " ?*tests-counter* " test(s):" crlf)
(do-for-all-facts ((?ttr_fact test_to_run)) TRUE
(funcall (fact-slot-value ?ttr_fact testsetupfunc))
(if (funcall (fact-slot-value ?ttr_fact testcheckfunc))
then (printout t " SUCCESS" crlf)
else (printout t " FAILURE" crlf)
(bind ?*failed-tests-counter* (+ 1 ?*failed-tests-counter*))
(bind ?*all-tests-passed* FALSE)))
(if ?*all-tests-passed*
then (printout t "All " ?*tests-counter* " tests passed successfully." crlf)
else (printout t ?*failed-tests-counter* "/" ?*tests-counter* " tests failed." crlf)))
tests\test_R1.clp:
;;; File: test_R1.clp
;;; Tests for Rule 1
(deffunction R1_TEST_1_SETUP ()
(load* "FluidSystem_facts_demo.clp")
(load* "FluidSystem_rules_demo.clp")
(reset))
(deffunction R1_TEST_1 ()
(send [JacketWaterInletTempReading] put-hasValue 35.0)
(send [JacketWaterInletTempReading] put-hasValueDefined DEFINED)
(send [JacketWaterOutletTempReading] put-hasValue 37.0)
(send [JacketWaterOutletTempReading] put-hasValueDefined DEFINED)
(run)
(return (member$ [DissimilarHighTempFlowRate] (send [CounterFlowHeatExchanger] get-hasIssue))))
test_all.clp:
;;; File: test_all.clp
;;; Run tests via:
;;; CLIPSDOS64.exe -f2 .\test_all.clp
(load* "TestingFramework.clp")
(setup-tests)
;;; Test R1
(load* "tests\\test_R1.clp")
(add_test (test_to_run "R1_TEST_1" R1_TEST_1_SETUP R1_TEST_1))
(clear) ;; unsure if this is needed
;;; ... more tests to follow
(run_all_tests)
(finish_tests)
The CLIPS regression tests use a framework that might serve your needs. You can download it (clips_feature_tests_640.zip) from one of the 6.4 download directories (sourceforge.net/projects/clipsrules/files/CLIPS/6.40_Beta_3/ for the current beta release). To run the tests, launch CLIPS in the same directory and execute a (batch "testall.tst") command. Using the CLIPS terminal application, you can also run them from the shell with "clips -f testall.tst". When execution completes you can look at the *.rsl files in the Results directory for the results. If difference occurs you can use a diff program to compare the contents of the .out file in the Actual directory with the contents of the Expected directory.
The framework uses the batch command to automatically load and run the test cases. The dribble-on command is used to capture the output of each test case and place it in its own file in the Actual directory. The framework allows you to run all of the test cases (using the testall.tst batch file) or you can run any of the individual test cases by running the .tst batch file associated with the test.
I have been creating some PCB footprints in KiCad recently, which are stored in s-expression files with data that looks like this:
(fp_text user %R (at 0 5.08) (layer F.Fab)
(effects (font (size 1 1) (thickness 0.15)))
)
(fp_line (start -27.04996 -3.986) (end -27.24996 -3.786) (layer F.Fab) (width 0.1))
(pad "" np_thru_hole circle (at 35.56 0) (size 3.175 3.175) (drill 3.175) (layers *.Cu *.Mask)
(clearance 1.5875))
(pad 96 smd rect (at 1.25 3.08473) (size 0.29972 1.45034) (layers F.Cu F.Paste F.Mask)
(clearance 0.09906))
I would like to be able to write shell one-liners to efficiently edit multiple parameters. I would normally use Awk for something like this, but the recursive nature of s-expressions makes it ill-suited for the task. I would like to know if there is a programming language with an interpreter designed to handle piped data and can process s-expressions natively. Perhaps a data-driven dialect of Lisp would do this, but I'm not sure where to look.
In summary, I would like to be able to make quick edits to an s-expression file in a similar manner to the way Awk lets me process columns of data line-by-line; only in the case of s-expressions the processing would be performed level-by-level.
Example: find all of the pad expressions of type smd with (size 0.29972 1.45034), and renumber each one based its position.
Simple script
Here is an example in Common Lisp, assuming your input is in file "/tmp/ex.cad" (it could also be obtained by reading the output stream of a process).
The main processing loop consists in opening the file in order to obtain an input stream in (which is automatically closed at the end of with-open-file), loop over all forms in the file, process them and possibly output them to standard output. You could complexify the process as much as you want, but the following is good enough:
(with-open-file (in #"/tmp/ex.cad")
(let ((*read-eval* nil))
(ignore-errors
(loop (process-form (read in))))))
Suppose you want to increase the width of fp_line entries, ignore fp_text and otherwise print the form unmodified, you could define process-form as follows:
(defun process-form (form)
(destructuring-bind (header . args) form
(print
(case header
(fp_line (let ((width (assoc 'width args)))
(when width (incf (second width) 3)))
form)
(fp_text (return-from process-form))
(t form)))))
Running the previous loop would then output:
(FP_LINE (START -27.04996 -3.986) (END -27.24996 -3.786) (LAYER F.FAB) (WIDTH 3.1))
(PAD "" NP_THRU_HOLE CIRCLE (AT 35.56 0) (SIZE 3.175 3.175) (DRILL 3.175) (LAYERS *.CU *.MASK) (CLEARANCE 1.5875))
(PAD 96 SMD RECT (AT 1.25 3.08473) (SIZE 0.29972 1.45034) (LAYERS F.CU F.PASTE F.MASK) (CLEARANCE 0.09906))
More safety
From there, you can build more elaborate pipelines, with the help of pattern matching or macros if you want. You have to take into account some safety measures, like binding *read-eval* to nil, using with-standard-io-syntax
and binding *print-circte* to T as suggested by tfb, disallowing fully qualified symbols (by having #\: signal an error), etc. Ultimately, like Shell scripts one-liners, the amount of precautions you add is based on how much you trust your inputs:
;; Load libraries
(ql:quickload '(:alexandria :optima))
;; Import symbols in current package
(use-package :optima)
(use-package :alexandria)
;; Transform source into a stream
(defgeneric ensure-stream (source)
(:method ((source pathname)) (open source))
(:method ((source string)) (make-string-input-stream source))
(:method ((source stream)) source))
;; make reader stop on illegal characters
(defun abort-reader (&rest values)
(error "Aborting reader: ~s" values))
Dedicated package for KiCad symbols (exporting is optional):
(defpackage :kicad
(:use)
(:export #:fp_text
#:fp_line
#:pad
#:size))
Loop over forms:
(defmacro do-forms ((form source &optional result) &body body)
"Loop over forms from source, eventually return result"
(with-gensyms (in form%)
`(with-open-stream (,in (ensure-stream ,source))
(with-standard-io-syntax
(let ((*read-eval* nil)
(*print-circle* t)
(*package* (find-package :kicad))
(*readtable* (copy-readtable)))
(set-macro-character #\: #'abort-reader nil)
(loop
:for ,form% := (read ,in nil ,in)
:until (eq ,form% ,in)
:do (let ((,form ,form%)) ,#body)
:finally (return ,result)))))))
Example:
;; Print lines at which there is a size parameter, and its value
(let ((line 0))
(labels ((size (alist) (second (assoc 'kicad:size alist)))
(emit (size) (when size (print `(:line ,line :size ,size))))
(process (options) (emit (size options))))
(do-forms (form #P"/tmp/ex.cad")
(match form
((list* 'kicad:fp_text _ _ options) (process options))
((list* 'kicad:fp_line options) (process options))
((list* 'kicad:pad _ _ _ options) (process options)))
(incf line))))
Output
(:LINE 2 :SIZE 3.175)
(:LINE 3 :SIZE 0.29972)
Just write a simple Lisp or Scheme script which loops on reading and processes recursively your s-expr as required. On Linux I would recommend using Guile (a good Scheme interpreter) or perhaps Clisp (a simple Common Lisp implementation) or even SBCL (a very powerful Common Lisp).
(You might consider DSSSL, but in your case it is overkill)
Notice that your sample input is not an S-expression, because (layer F.Fab) is not one (since after the dot you should have another s-expression, not an atom like Fab). I guess it is a typo and should be (layer "F.Fab"); or maybe your KiCad software don't process S-expressions, but some other input language (which should be specified, probably in EBNF notation) inspired by S-expressions.
Notice also that KiCad is a free software and has a community with forums and a mailing list. Perhaps you should ask your actual problem there?
PS. We don't know what transformation you have in mind, but Scheme and Common Lisp are really fit for such tasks. In most cases they are extremely simple to code (probably a few lines only).
It seems that Racket is incapable of reading a string from STDIN.
$ racket
Welcome to Racket v6.4.
-> (define (s) (read-line))
OK, s is an alias for a call to read-line.
-> (printf "You entered: ~a\n" s)
You entered:
Failure: The string is printed, but Racket does not wait for keypress / STDIN / EOF / EOL.
-> (define n (read))
a
-> n
'a
Failure: This makes a call to read and waits for EOF / EOL, then assigns to n, but n is assigned the symbol 'a not the string literal a.
-> (read-line)
""
Failure: calling read-line doesn't wait for STDIN, just returns the empty string.
-> (read-string 5)
asdasdasdasd
"\nasda"
; sdasdasd: undefined;
; cannot reference undefined identifier
; [,bt for context]
Failure: only reads 5 bytes of STDIN, and apparently evals the rest of it... ?
-> (read-string 500000)
asdasd
asdasdaas
a
asdasd
asdasd
asdasd
Failure: doesn't return until exactly 500000 bytes have been read, and doesn't return on EOL.
Somewhat like Python's input() which returns a string when EOL is found, or Factor's readln which does the same, how can I read raw data from the current-input-port?
This is just a limitation of the Racket's REPL's input handling. If you write a standalone program it will work fine.
Here's a quote from the mailing list that explains the problem:
A known limitation. The REPL implemented by plain `racket' does not
separate the input stream for REPL expressions from the program's
input stream.
More details: https://groups.google.com/d/msg/racket-users/0TTsA9-3HDs/9_mMWsgKFOMJ
I got a big file and I would like to replace the first line with other content.
When I use {ok, IoDev} = file:open("/root/FileName", [write, raw, binary]), the whole content is removed.
But when I use {ok, IoDev} = file:open("/root/FileName", [append, raw, binary]) and file:pwrite(S, {bof,0}, <<"new content\n">>), I got the result {error, badarg}.
If I set Location to 0: file:pwrite(S, 0, <<"new content\n">>), the string is appended at tail of the file.
You seem to be confused with the actual file API.
file:open/2 will truncate the file if you pass [write, raw, binary]as you do:
(about write mode): The file is opened for writing. It is created if it does not exist. If the file exists, and if write is not combined with read, the file will be truncated.
So you need to pass either [write, read] or [write, append] as documented.
file:pwrite/3 also works exactly as documented. It allows you to write at a given position in the file. In particular, you cannot pass {bof, 0} as second argument since you opened the file in raw mode:
If IoDevice has been opened in raw mode, some restrictions apply: Location is only allowed to be an integer; and the current position of the file is undefined after the operation.
The following sample code shows how they work:
ok = file:write_file("/tmp/file", "This is line 1.\nThis is line 2.\n"),
{ok, F} = file:open("/tmp/file", [read, write, raw, binary]),
ok = file:pwrite(F, 0, <<"This is line A.\n">>),
ok = file:close(F),
{ok, Content} = file:read_file("/tmp/file"),
io:put_chars(Content),
ok = file:delete("/tmp/file").
It will output:
This is line A.
This is line 2.
This works because text "This is line A.\n" is exactly as long as "This is line 1.\n". It does not really replace the line, but just bytes. If you need to replace the first line with content that has a different length, you need to rewrite the whole content of the file. A common approach is indeed to write a new file and swap them eventually. If the file is small enough, however, you can read it entirely in memory and rewrite it. file:read_file/1 and file:write_file/2 would work:
replace_first_line(Path, NewLine) ->
{ok, Content} = file:read_file(Path),
[FirstLine | Tail] = binary:split(Content, <<"\n">>),
NewContent = [NewLine, <<"\n">> | Tail],
ok = file:write_file(Path, NewContent).
The question is not related to erlang but rather general file operations.
Replacing a line in a file requires to rewrite the file in a whole. The easiest way to do so would be to write all the new content in a new file and then to move the file.
Consider this code:
OperatorTable addOperator(":", 2)
: := method(value,
list(self, value)
)
hash := "key": "value"
hash println
The return should be list(key, value), and when using this in the Io REPL that is exactly the return value. When using the interpreter (as in io somefile.io) the value returned is value. After some inspection the difference is here:
# In the REPL
OperatorTable addOperator(":", 2)
message("k" : "v") # => "k" :("v")
# Via the Interpreter
OperatorTable addOperator(":", 2)
message("k" : "v") # => "k" : "v"
Why is this happening?
File execution happens in these stages:
load file
replace operators based on the current operator table
execute contents
So operator to message conversion only happens when the file is initially loaded in stage 2.
When the operator registration code is executed in stage 3. this has already happened,
thus the operator has no effect.
You can set the order which files get loaded manually and put the operator definition in the first file loaded.
Having a file called operators.io for example which includes all operator definitions loaded before the files that use them.
After confirming with ticking I arrived at the following solution:
main.io:
doFile("ops.io")
doFile("script.io")
ops.io:
OperatorTable addOperator(":", 2)
: := method(value,
list(self, value))
script.io:
hash := "key": "value"
hash println
Like ticking explains, the whole file is loaded at once so you have to split it up so the loading order guarantees that the operators are available.