Replace regex in a file, in-place, with Common Lisp - file-io

I am trying to write the Common Lisp version of Python's regex search and replace, with in-place modification of files:
import fileinput, re
for line in fileinput.input(inplace=1, backup='.bak'):
line = re.sub(r"foo", "bar", line, re.M)
print (line)
This is the Common Lisp code I was able to think up:
(require :cl-ppcre)
(defun in-place-subst (file)
(with-open-file (stream file :direction :io :if-exists :overwrite)
(loop for line = (read-line stream nil)
while line do
(write-line (cl-ppcre:regex-replace-all "foo" line "bar") stream))))
It works, sort of. Right now the replacement text will be appended at the end of the file. My immediate problem is that I can't figure out how to replace the content.
To better explain, if file.txt contains:
1 foo
2 bar
3 foobar
after calling
(in-place-subst "file.txt")
I get:
1 foo
2 bar
3 foobar
1 bar
2 bar
3 barbar
Instead of the right replacement:
1 bar
2 bar
3 barbar
I tried with all possible with-open-file options (from Successful Lisp), with no success:
Keyword Value Action if File Exists
---------- ------------------ ---------------------------------------
:IF-EXISTS NIL return NIL
:IF-EXISTS :ERROR signal an error
:IF-EXISTS :NEW-VERSION next version (or error)
:IF-EXISTS :RENAME rename existing, create new
:IF-EXISTS :SUPERSEDE replace file upon CLOSE
:IF-EXISTS :RENAME-AND-DELETE rename and delete existing, create new
:IF-EXISTS :OVERWRITE reuse existing file (position at start)
:IF-EXISTS :APPEND reuse existing file (position at end)
Could somebody please send me in the right direction, so that the function will render file.txt in the correct way?
Also, what would be the Common Lisp idiomatic way for doing this, assuming of course cl-ppcre is available?
Is there a more succinct way of doing an in-place regex substitution with Common Lisp?

In Python there is no primitive operation that modifies a file “in-place”; instead,
there is a function of an helper class fileinput, that gives the illusion of
modifying a file in place by first copying the file to a backup file, and then
reading the backup file and writing the result of processing it to the original one. From the manual:
Optional in-place filtering: if the keyword argument inplace=1 is passed
to fileinput.input() or to the FileInput constructor, the file is moved
to a backup file and standard output is directed to the input file
(if a file of the same name as the backup file already exists,
it will be replaced silently).
This makes it possible to write a filter that rewrites its input file in place.
If the backup parameter is given (typically as backup='.'),
it specifies the extension for the backup file, and the backup file remains
around; by default, the extension is '.bak' and it is deleted when the output
file is closed. In-place filtering is disabled when standard input is read.
So, the way of doing this operation in Common Lisp is to mimic the
Python code, by first copying the file to a backup file, for instance using this function my-copy-file, then writing the following code:
(defun in-place-subst (file)
(let ((backup-file (concatenate 'string file ".bak")))
(my-copy-file file backup-file)
(with-open-file (in-stream backup-file)
(with-open-file (out-stream file :direction :output :if-exists :supersede)
(loop for line = (read-line in-stream nil)
while line do
(write-line (cl-ppcre:regex-replace-all "foo" line "bar") out-stream))))))

Related

AHK: How to store a selected file in a variable to execute an script using FileSelectFile function after selected in Window/File explorer?

Apologies, I am quite new with AHK.
Context: I am trying to build a small program (eventually with UI) which will clean data in .XLF files in order to be processed properly by a CAT tool interpreter (import into it).
By "clean" I mean to find HTML attributes and replace them with their respective Char Entities. This as a single script; writing the name of the file or path inside the script is working perfectly.
Problem: I would like to run my .ahk/.exe allowing the user to open the file manager/explorer and select the file that needs to be processed by the script (find/replace html attributes with char entities) selecting the file is not working. Nothing is populated (the final file/result is empty) I'm trying to sort out this with FileSelectFile function and store the output var value (selecting the file) in the first instruction "fileread, selectedfile".
But it's not working! If I don't do this and I only provide in the default directory "A_ScriptDir" an specific file name .xf -> this works fine.
This is my code so far w/comments:
SetWorkingDir, %A_ScriptDir%
FileEncoding, UTF-8
;NoEnv
;Open Window File Manager/Explorer and select a file .xlf file
FileSelectFile, SelectedFile, 8, , Open a file, , ,(*.xlf)
;--- > HTML attribute '&' must be replaced by its char entity first/overall otherwise this instruction will overwrite the amp entities from the rest of char entities corrupting the file;
;"SelectedFile" can be any filename such as "example.xlf" but this is not my scope
fileread, text, SelectedFile ;previously "text.xlf" with random html content to do tests
replace := "&"
newtext := strreplace(text, "&", replace, all)
sleep, 200
filedelete, newtest.xlf
fileappend, %newtext%, newtest.xlf
;--------------------------------- <b>
;here "fileread" must read the final appended file as solution to use "streplace" function multiple times (replace more than one desired string) running the script at the same time. (due to my lack of exp. with loop function)
fileread, text, newtest.xlf
replace := ">b<"
newtext := strreplace(text, "<b>", replace, all)
filedelete, newtest.xlf
fileappend, %newtext%, newtest.xlf
I've been thinking that other solution can be:
I am still new to understand apply Drag and Drop GUI but I am unsure how to modify my code in order to drag/drop a file onto the ahk.exe
Thanks for your time reading this! any tip and/or help would be super appreciated :)
The FileRead command expects text, not an input variable.
So if you add % around SelectedFile like this, it should work:
FileRead, text, %SelectedFile%
If that doesn't work, it means the file does not exist or an error occurred. In that, you'll want to look at FileRead's error handling section.

Scheme Macro - Adding a Header to a file

I am writing a scheme macro for a simulation tool. I create thousands of files and I want to add a header (6 lines) to each file. The code for my header runs well and the header gets created in the right way.
But the adding of the header to my file is buggy. It does not add the 6 header lines to top of my files without touching the rest, it delets the first information that are in my file. How much information is deleted, depends on the total length of header-information.
(let* ((out (open-input-output-file filename0) ))
(display header out)
(newline out)
(close-output-port out))
This is how my file looks without the header:
TracePro Release: 20 6 0
Irradiance Map Data for D:****\TracePro\Aktive\sim_mod_09.oml
Linear Units in millimeters
Data for absorbing_area_focuscircle Surface 0
Data generated at 10:55:56 May 28, 2021
This is how my file looks with the header:
axle x y z a b c
pyra 0 0 0 0 0 0
lens 0 0 0 0 0 0
coll 0 0 0 0 0 0
mir1 0 0 0 0 0 0
glass1 0 0 0 0 0 0
ing_area_focuscircle Surface 0
Data generated at 10:57:29 May 28, 2021
Raytrace Time: mins: 0, secs: 0*
Projected Plane Extent from surface geometry
TopLeft:(-1.05125,-214.755,-1.05125)
TopRight:(1.05125,-214.755,-1.05125)
BottomLeft:(-1.05125,-214.755,1.05125)
BottomRight:(1.05125,-214.755,1.05125)
This isn't really an answer, especially as my knowledge of the Scheme language standards isn't good enough to even know if this is possible within a strictly-defined Scheme. However I'll show why it's hard, and then give an example of how to solve it in Racket, first by cheating to make a probably-correct answer and then by trying to do it the hard way to make a probably-not-correct answer.
Why it is hard
No modern filesystem I know of allows you to open a file for 'insertion' where new content is inserted into the file, pushing existing content 'down' the file. Instead you can open a file for writing, conceptually, in two ways:
for appending, which will append new content at the end;
for overwriting, which will overwrite existing content with new.
(Actually these may be the same: opening for appending may just open for overwriting and then move the current location to the end of the file.)
So what you're doing in your sample is opening for overwriting, and then clobbering the content of the file with the header.
How to do it, in outline
The way to do what you need to do, in outline, is:
create and open a temporary file in the same directory as the file you care about;
write the new content to the temporary file;
copy all the content of the existing file to the temporary file;
close the temporary file;
if all is well, rename the temporary file on top of the existing file, if all is not OK, delete it.
If you do this carefully it is safe, because file renames are atomic, or should be, in the filesystem, if the two files are in the same directory. That means that the rename should either completely succeed or completely fail, even if the system crashes part way through or the filesystem fills or something like that. If the filesystem doesn't garuantee that then you're pretty much stuck.
But doing it carefully is not easy (I should admit here that some of my background is doing things like this to system-critical files, so I've spent too long thinking about how to make this safe in a context where getting it wrong is very serious indeed).
Solving this in Racket by cheating
As I said, getting the above process right is hard, and it is therefore something you often want to rely on a battle-tested library for. Racket has such a thing: call-with-atomic-output-file. This seems to be designed to solve exactly this problem: it deals with creating and opening the temporary file for you, deals with the renaming at the end and cleans up appropriately. All you need is a function which copies things around.
So here is a function, prepend-to-file which uses call-with-atomic-output-file to try and do what you want. This is Racket-specific, in many ways, and it is also somewhat overengineered.
(define (prepend-to-file file content #:buffer-size (buffer-size 40960))
;; prepend content to file
;;
;; Try to be a bit too clever about whether we're copying strings or bytes
;; based on the argument
(let-values ([(read-it! write-it make-it)
(if (bytes? content)
(values read-bytes! write-bytes make-bytes)
(values read-string! write-string make-string))])
(call-with-atomic-output-file file
(λ (out path)
;; out is open for writing to the temporary file, path is the
;; temporary file's pathname which we don't care about
(call-with-input-file file
(λ (in)
;; in is now open for reading from the real file
(write-it content out)
(let ([buffer (make-it buffer-size)])
;; copy in to out using a buffer
(for ([nread (in-producer (thunk
(read-it! buffer in))
eof-object?)])
(write-it buffer out 0 nread)))
;; OK just return the file for want of anything better
file))))))
I think it's reasonably likely that the above code actually works in most reasonable cases.
Solving this in Racket without cheating
If we could write call-with-atomic-output-file then we could solve the problem without cheating. But getting this right is hard. Here is an attempt to do this, which is almost certainly incorrect:
(define (call/temporary-output-file file proc)
(let ([tmpname (string-append file
"-"
(number->string (random (expt 2 24))))]
[managed #f]
[once #t])
;; tmpname is the name of the temporary file: this assumes pathnames are
;; only strings, which is wrong. managed is a flag which tells us if
;; proc returned normally, once is a flag which attempts to prevent any
;; continuation nasties so the whole thing can only happen once.
(call-with-output-file tmpname
(λ (out)
(dynamic-wind
(thunk
(when (not once)
;; if this is the case we're getting back in, and this
;; is not OK
(error 'call/temporary-output-file
"this is hopeless")))
(thunk
;; call proc and if it returns normally note that
(begin0 (proc out tmpname)
(set! managed #t)))
(thunk
;; close the output port regardless
(close-output-port out)
(if managed
;; We did OK, so rename the file in place
(rename-file-or-directory tmpname file #t)
;; failed, nuke the temporary file
(when (file-exists? tmpname)
(delete-file tmpname)))
;; finally set once false to prevent shenanigans
(set! once #f)))))))
Notes:
this is still Racket-specific, but it now depends only on simpler functions which have, probably, more obvious counterparts in other implementations (or in the standard);
it tries to deal with some of the edge cases, but almost certainly misses some;
it certainly does not cope in cases such as the rename failing and so on;
Again: don't use this: it's almost certainly buggy.
However if you did use this, then you could simply splice it in instead of call-with-atomic-output-file in the above code and it will, often but probably not always, work.

While read loop and command with file output

I have run into an issue making a while loop (yes, I am new at this..).
I have a file $lines_to_find.txt, containing a list of names which I would like to find in another (large) file $file_to_search.fasta.
When the lines in lines_to_find.txt are found in file_to_search.fasta, the lines with search hits I would like to be printed to a new file: output_file.fasta.
So I have a command similar to grep, that takes the sequences (for that is whats in the large file), and prints them to a new file:
obigrep -D SEARCHWORD INPUTFILE.fasta > OUPUTFILE.fasta
Now I would like the searchword to be replaced with the file lines_to_find.txt, and each line should be read and matched to the file_to_search.fasta. Output should preferably be one file, containing the sequence-hits from all lines in file lines_to_find.txt.
I tried this:
while read line
do
obigrep -D '$line' file_to_search.fasta >> outputfile.fasta
done < lines_to_find.txt
But my outputfile just returns empty.
What am I doing wrong?
Am I just building the while read loop wrong?
Are there other ways to do it?
I'm open to all suggestions, and as I am new, please point out obvious begginer-flaws.

how to fetch a data from one file location and to run using tcl code

In tcl how to get the data from one file location and to run that data using TCL code .
for example
In the folder 1 there is config file ,i want to get the informations of config file and i want to execute the information that is present or not,
If the configuration file contains Tcl code, it's just:
# Put the filename in quotes if you want, or in a variable, or ...
source /the/path/to/the/file.tcl
If the file contains Tcl code but you don't trust it, you can use a “safe interpreter” context. This disables many commands, giving a much more restricted set of capabilities that you can then add specific exceptions to (with interp alias):
# Make the context
set i [interp create -safe]
# Set up a way for the context to let the master find out about what to
# really set
interp alias $i configure {} recordConfiguration
proc recordConfiguration args {
puts "configured with $args"
}
# Evaluate the script (note that [source] is hidden by default) in the context
$i invokehidden source /the/path/to/the/file.tcl
# Dispose the context
interp delete $i
If the file isn't Tcl code, you have to parse it. That's a substantially more complex matter, so much so that we'll need to know the format of the file before we can answer.
If you are trying to read data (like text strings) from a file then you'll have to open a channel for that particular file like this:
set fileid [open "path/to/your/file.txt" r]
Read open manual page.
Then you can use gets command to read data from the file through the channel fileid .

Why is read-line run twice for reading from a file in Lisp?

This is the code to implement the 'cat' command with lisp, as is explained in the book ANSI Common Lisp, page 122.
(defun pseudo-cat (file)
(with-open-file (str file :direction :input)
(do ((line (read-line str nil 'eof)
(read-line str nil 'eof)))
((eql line 'eof))
(format t "~A~%" line))))
Why is the read-line function run twice? I tried to run it with only one read-line, but the Lisp couldn't finish the code.
The syntax of DO variables is: variable, initialization form, update form. In this case, the initialization form is the same as the update form. But there is no shorthand for that case in DO, so you have to write it out twice.
You need to read the syntax of DO: http://www.lispworks.com/documentation/HyperSpec/Body/m_do_do.htm
The first READ-LINE form is the init-form and the second is the step-form. So in the first iteration the variable is set to the result of the init-form. In the next iterations the variable is set to the value of the step-form.
You can use (listen file) for test if you can read from the file.
This is my print-file function
(defun print-file (filename)
"Print file on stdout."
(with-open-file (file filename :direction :input)
(loop
(when (not (listen file)) (return))
(write-line (read-line file)))))