Finding the attributes of Chinese filenames using NewLISP? - utf-16

The following NewLISP code shows me the file attributes of files under Win32. However, some of the filenames retrieved have Chinese characters in the name. When the GetFileAttributesA function encounters them, it gives me a -1 for the attribute. I looked at GetFileAttributesW but don't know how to make the contents of the fname available to the function in a form it recognises.
How does one handle this situation? (I am willing to consider trying another language)
(define (get-archive-flag file-name)
(if (not GetFileAttributesA)
(begin
(import "kernel32.DLL" "GetFileAttributesA")
)
)
(setq fname file-name file-attrib (GetFileAttributesA (address fname)))
(append fname " " ( string file-attrib))
)
; walks a disk directory and prints all path-file names
;
(define (show-tree dir)
(if (directory dir)
(dolist (nde (directory dir))
(if (and (directory? (append dir "/" nde))
(!= nde ".") (!= nde ".."))
(show-tree (append dir "/" nde))
(println (get-archive-flag (append dir "/" nde)))
)
)
)
)
(show-tree "z:\\working files\\Cathy")

Since version 10.3.2, relased July 20th 2011, 10.3.2 , MultiByteToWideChar is handled internally by newLISP when reading path/file names.

Maybe you do not use Unicode version of Newlisp. Anycase, there are very few Newlispers here. Try Newlisp forum instead.

For the sake of completeness, here's the solution, found in consultation with the folk on the NewLISP forum.
I haven't replaced the bits slice reverse technique on the attribute bit with the more appropriate & operator. That's left for the reader.
(constant 'SIZEOF_WCHAR 2) ; assumption
(constant 'CP_UTF8 65001)
(define (utf8->16 lpMultiByteStr , cchWideChar lpWideCharStr ret)
(if (not MultiByteToWideChar)
(begin
(import "kernel32.DLL" "MultiByteToWideChar")
)
)
; calculate the size of buffer (in WCHAR's)
(setq cchWideChar
(
MultiByteToWideChar
CP_UTF8 ; from UTF-8
0 ; no flags necessary
lpMultiByteStr
-1 ; convert until NULL is encountered
0
0
)
)
; allocate the buffer
(setq lpWideCharStr (dup " " (* cchWideChar SIZEOF_WCHAR)))
; convert
(setq ret
(
MultiByteToWideChar
CP_UTF8 ; from UTF-8
0 ; no flags necessary
lpMultiByteStr
-1 ; convert until NULL is encountered
lpWideCharStr
cchWideChar
)
)
(if (> ret 0) lpWideCharStr nil)
)
; resets the Win32 archive flag on a file
; By CaveGuy 2009
(define (get-archive-flag file-name)
(if (not GetFileAttributesW)
(begin
(import "kernel32.DLL" "GetFileAttributesW")
)
)
(setq fname file-name
file-attrib (GetFileAttributesW (utf8->16 fname))
)
file-attrib
)
; walks a disk directory and prints all path-file names where archive bit is set
;
(define (show-tree dir)
(if (directory dir)
(dolist (nde (directory dir))
(if (and (directory? (append dir "/" nde)) (!= nde ".") (!= nde "..") )
(show-tree (append dir "/" nde))
(if (not (or (= nde ".") (= nde "..")))
(begin
(setq fname (append dir "/" nde))
(setq fflag (get-archive-flag fname))
(setq fbits (bits fflag))
(if (= (slice (reverse fbits) 5 1) "1") (println fname))
)
)
)
)
)
)
(show-tree "//server/folder")

Related

Why `optional` in a Parser can err out

https://github.com/complyue/dcp is a minimum working example to reprod this error
$ cabal run dcp:dcp < samples/basic.txt
Up to date
dcp: 10:1:
|
10 | method doXXX() pass
| ^
unexpected 'm'
expecting ';'
CallStack (from HasCallStack):
error, called at src/Parser.hs:149:14 in main:Parser
$
I believe it's optionalSemicolon causing the failure:
https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L50-L54
findIt = do
-- ignore leading whitespaces and an optional semicolon in between
nbsc >> optionalSemicolon >> nbsc
-- try get a doc comment block
getIt >>= \case
And it's defined like this:
https://github.com/complyue/dcp/blob/1df7ad590d78d4fa9a017eb53f9f265e291bdfa7/src/Parser.hs#L31-L32
optionalSemicolon :: Parser Bool
optionalSemicolon = fromMaybe False <$> optional (True <$ symbol ";")
I can't reason about why it can fail like this.
Turns out it is because the symbol in optionalSemicolon is referencing sc while it shouldn't. Solved like this:
https://github.com/complyue/dcp/commit/fd2df02f7218e59db2a732d5de74acedfefefaa2?branch=fd2df02f7218e59db2a732d5de74acedfefefaa2&diff=unified#diff-1785501875711a0bda12ba99505aa188659de4e35ad27d7fb819993bd8ec95bdL26-R145
optionalComma :: Parser Bool
-optionalComma = fromMaybe False <$> optional (True <$ symbol ",")
+optionalComma = fromMaybe False <$> optional (True <$ string ",")
optionalSemicolon :: Parser Bool
-optionalSemicolon = fromMaybe False <$> optional (True <$ symbol ";")
+optionalSemicolon = fromMaybe False <$> optional (True <$ string ";")

How to read a data file with some condition faster in Fortran?

I am trying to write down a Fortran subroutine for my code in order to read a data from a file (which is a huge data set on itself).The data file contains the Location (nx0,ny0,nz0) and the field related to that location (Bx,By,Bz).
(Ex: lets say the range for nx0, ny0 and nz0 is from [-15,15].
so the number of rows will be 31*31*31=29791)
-15.00000 -15.00000 -15.00000 700.00000 -590.00000 100.00000
-15.00000 -15.00000 -14.00000 -110.00000 -570.00000 100.00000
-15.00000 -15.00000 -13.00000 -550.00000 -200.00000 100.00000
-15.00000 -15.00000 -12.00000 -540.00000 -230.00000 100.00000
-15.00000 -15.00000 -11.00000 -140.00000 -50.00000 100.00000
. . . . . .
. . . . . .
. . . . . .
15.00000 15.00000 15.00000 140.00000 50.00000 100.000
What I want to do is to look for a specific location within my file (xi,yi and zi) and read the field related to that location then use it for further analysis. Not only the related field to the target position itself but also the surrounding field of that location (Like the three other side of the square around the target point).
subroutine read_data(xi,yi,zi,Bxij,Byij)
real*8,intent(in) :: xi,yi,zi !,time
real*8,intent(out) :: Bxij(4),Byij(4) !,Bzij(4)
integer,parameter :: step = 1 ,cols = 6, rows = 29791 !!!15,000,000
real,dimension(rows) :: nx0,ny0,nz0,Bx,By,Bz
character*15 filein
character*35 path_file
path_file = '/home/mehdi/Desktop/'
filein= 'test-0001'
open(7,file=trim(path_file)//filein, status='old',action='read')
xi_1 = xi +step
yi_1 = yi +step
do i = 1,rows
read(7,*) nx0(i),ny0(i),nz0(i),Bx(i),By(i),Bz(i)
c
if ( xi == nx0(i) .and. yi == ny0(i) .and.
& zi == nz0(i)) then
Bxij(1) = Bx(i)
Byij(1) = By(i)
cycle
endif
c
if ( xi == nx0(i) .and. yi_1 == ny0(i) .and.
& zi == nz0(i)) then
Bxij(2) = Bx(i)
Byij(2) = By(i)
cycle
endif
c
if ( xi_1 == nx0(i) .and. yi == ny0(i) .and.
& zi == nz0(i)) then
Bxij(3) = Bx(i)
Byij(3) = By(i)
cycle
endif
c
if ( xi_1 == nx0(i) .and. yi_1 == ny0(i) .and.
& zi == nz0(i)) then
Bxij(4) = Bx(i)
Byij(4) = By(i)
exit
endif
c
close(7)
enddo
end
I have done it this way but it is too slow. One of the most important things for me is the speed (which even for this small fraction of data set is really time consuming).
I know this slow mode is for the needs to read the whole data set each time in order to look for the target points. This subroutine is called couple times within the code and for the further steps the code is going to do the same thing over and over again, so it is time consuming.
How can I make this code work more efficiently?
Before I begin this answer, let me reiterate what I said in the comments to your question:
Do not underestimate how much data you can put into a single array. Reading once, and then having everything in memory is still the fastest way possible.
But let's assume that the data really gets too big.
Your main issue seems to be that you have to re-read all the data from the beginning until you find the value you're looking for. That takes the time.
If you can calculate which line of the data file the value you are interested in is, it might help to convert the file into an unformatted direct access file.
Here is an example code for the conversion. It's using Fortran 2008 features, so if your compiler can't do it, you have to modify it:
program convert
use iso_fortran_env, only: real64
implicit none
integer, parameter :: reclength = 6*8 ! Six 8-byte values
integer :: ii, ios
integer :: u_in, u_out
real(kind=real64) :: pos(3), B(3)
open(newunit=u_in, file='data.txt', form='formatted', &
status='old', action='read', access='sequential')
open(newunit=u_out, file='data.bin', form='unformatted', &
status='new', action='write', access='direct', recl=reclength)
ii = 0
do
ii = ii + 1
read(u_in, *, iostat=ios) pos, B
if (ios /= 0) exit
write(u_out, rec=ii) pos, B
end do
close(u_out)
close(u_in)
end program convert
Once you have converted the data, you can read only the record you need, as long as you can calculate which one it is. I have assumed that just like in your example, the z-coordinate changes fastest and the x-coordinate changes slowest.
program read_txt
use iso_fortran_env, only: real64
implicit none
integer, parameter :: nx=601, ny=181, nz=61
real(kind=real64), parameter :: x_min=real(-nx/2, kind=real64)
real(kind=real64), parameter :: y_min=real(-ny/2, kind=real64)
real(kind=real64), parameter :: z_min=real(-nz/2, kind=real64)
real(kind=real64), parameter :: x_step = 1.0_real64
real(kind=real64), parameter :: y_step = 1.0_real64
real(kind=real64), parameter :: z_step = 1.0_real64
real(kind=real64) :: request(3), pos(3), B(3)
integer :: ios, u_in
integer :: ii, jj, kk, record
integer, parameter :: reclength = 6 * 8 ! Six 8-byte values
open(newunit=u_in, file='data.bin', access='direct', form='unformatted', &
status='old', action='read', recl=reclength)
mainloop : do
read(*, *, iostat=ios) request
if (ios /= 0) exit mainloop
write(*, '(A, 3F7.2)') 'searching for ', request
! Calculate record
ii = nint((request(1)-x_min)/x_step)
jj = nint((request(2)-y_min)/y_step)
kk = nint((request(3)-z_min)/z_step)
record = kk + jj * nz + ii * nz * ny + 1
read(u_in, rec=record, iostat=ios) pos, B
if (ios /= 0) then
print *, 'failure to read'
cycle mainloop
end if
write(*, '(2(A, 3F7.2))') "found pos: ", pos, " Bx, By, Bz: ", B
end do mainloop
close(u_in)
end program read_txt
Note that the unformatted is not compiler- and system independent. A file created on one computer or with a program compiled by one compiler might not be able to be read with another program or on another computer.
But if you have control over it, it might be a useful way to speed things up.
PS: I left the x, y, and z coordinates in the file so that you can check whether the values are actually what you wanted. Always good to verify these things.

Removing binary zero from a string

I've had to move a whole directory structure (thousands of directories and files) from the Mac to the PC, but now on the PC for some reason some of the folder names have a character on the end of value binary zero ( a few have this in the middle). I need to clean this up since it crashes a macro which tries to read these directories.
I've tried the following code in a vba-word macro using the Replace function (as part of a larger program that walks through the directory tree) but the Replace function doesn't seem to catch chr(0) .
Set current_folder = fso.GetFolder(source_folder.Path)
current_folder_name = current_folder.Name
cleaned_folder_name = Replace(current_folder_name, Chr(0), " ")
Selection.TypeText "Old directory name: " & current_folder_name
Selection.TypeParagraph
Selection.TypeText "New directory name: " & cleaned_folder_name
Selection.TypeParagraph
If current_folder_name <> cleaned_folder_name Then
current_folder.Name = cleaned_folder_name
End If
I've also tried:
cleaned_folder_name = Replace(current_folder_name, Chr(0), " ", 1, -1, vbBinaryCompare)
How I can get the Replace function to replace binary 0 in a string by a blank.
Or does anyone knows a different approach to cleaning up these directory and file names that would work.
Thanks,
Harry
This should do it:
Dim OldFolderName As String, NewFolderName As String
OldFolderName = source_folder.Path
If InStr(OldFolderName, Chr(0)) > 0 Then
'NewFolderName = Application.Clean(OldFolderName) 'Works on some versions
NewFolderName = Application.WorksheetFunction.Clean(OldFolderName) 'By Gene Skuratovsky
Name OldFolderName As NewFolderName
End If
Edit2: Probably best to use the Clean() method.
In my case Replace() and InStr() don't work with Chr(0) (can't see it not displaying any error), but this removal can be done for example this way:
If Mid$(str, 1, 1) = Chr$(0) Then str = Mid$(str, 2)
If Mid$(str, Len(str), 1) = Chr$(0) Then str = Left$(str, Len(str) - 1)
This is Null removal from ends, for example from objFile.ExtendedProperty("Dimensions") value, like "?600 x 400?". Nulls (?) are insterted here in Windows 10 but not in Windows XP.
You can write a custom loop function to determine what character you are having issues with:
OldFolderName = source_folder.Path
For x = 0 To 255
If InStr(OldFolderName, Chr(x)) > 0 Then
MsgBox "Chr(" & x & ") is found at position " & InStr(OldFolderName, Chr(x))
EndIf
Next x

iTunes method SaveArtworkToFile failed, called from AutoHotkey COM object

I'm trying to extract the artwork file from my iTunes MP3 files using AutoHotkey (v1.1). The script works well until it gets to the SaveArtworkToFile method.
objITunesApp := ComObjCreate("iTunes.Application")
objLibrary := objITunesApp.Sources.Item(1)
objPlaylist := objLibrary.Playlists.ItemByName("! iTunesCovers")
objTracks := objPlaylist.Tracks
Loop, % objTracks.Count
{
objTrack := objTracks.Item(A_Index)
Loop, % objTrack.Artwork.Count
{
objArtwork := objTrack.Artwork.Item(A_Index)
TrayTip, % "Track Index: " . objTrack.index
, % "Artwork: " . A_Index . "/" . objTrack.Artwork.Count . "`n"
. "Format: " . objArtwork.Format . "`n"
. "IsDownloadedArtwork: " . objArtwork.IsDownloadedArtwork . "`n"
. "Description: " . objArtwork.Description
strFilePath := objTrack.index . "-" . A_Index
if (objArtwork.Format = 1)
strExtension := "bmp"
else if (objArtwork.Format = 2)
strExtension := "jpg"
else if (objArtwork.Format = 4)
strExtension := "gif"
else if (objArtwork.Format = 5)
strExtension := "png"
else
strExtension := ""
strResult := objArtwork.SaveArtworkToFile(strFilePath . "." . strExtension)
MsgBox, % strFilePath . "." . strExtension . "`nResult: " . strResult
}
}
I get this error message:
---------------------------
SaveArtworkToFile.ahk
---------------------------
Error: 0x8000FFFF - Défaillance irrémédiable
Source: (null)
Description: (null)
HelpFile: (null)
HelpContext: 0
Specifically: SaveArtworkToFile
Line#
---> 017: strResult := objArtwork.SaveArtworkToFile(strFilePath)
---------------------------
I have the same result with bpm and jpg file formats. And strResult returned by SaveArtworkToFile is empty. Should this method be supported by the AHK iTunes.Application COM object?
Thanks and Happy New Year!
#Manuell: Oh! Thanks for putting back the doc to my attention. In the
Parameters: filePath Full path to the artwork image file.
I missed the word "Full". In my script, I was relying on relative path. I just tested it with an absolute path and this work!
Googled it for you: IITArtwork::SaveArtworkToFile
HRESULT IITArtwork::SaveArtworkToFile ( [in] BSTR filePath )
Save artwork data to an image file.
The format of the saved data is specified by the artwork's format
(JPEG, PNG, or BMP). The directory that contains the file must already
exist, it will not be created. If the file already exists, its
contents will be replaced.
Parameters: filePath Full path to the artwork image file.
That method doen't return a value (as Hans said in comment). Try:
objArtwork.SaveArtworkToFile(strFilePath . "." . strExtension)

Lua Script Pattern Matching Problem

First of all, I have been using this site as a reference through the entire scripting process and it has been wonderful. I appreciate how useful and knowledgeable everyone is here. With that in mind, I have a question regarding matching (pattern matching) in Lua. I am writing a script that essentially takes input from a file and imports it into a table. I am checking for specific MAC addresses in the file as the host I am querying.
if macFile then
local file = io.open(macFile)
if file then
for line in file:lines() do
local f = line
i, j = string.find ( f, "%x+" )
m = string.sub(f, i, j)
table.insert( macTable, m )
end
file:close()
end
This parses the file into a format I will use to query later. Once the table is built, I run a pattern matching sequence to try and match the MAC from the table by iterating the table and matching the pattern against the current iteration:
local output = {}
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
for key,value in next,macTable,nil do
a, p = string.find ( s, value )
matchFound = string.sub(s, a, p)
table.insert( output, matchFound )
end
This doesn't return any output although when I enter it line by line in a Lua prompt, it seems to work. The variables are being passed correctly I believe. Any suggestions?
If your macFile uses a structure like this:
012345678900
008967452301
000000000000
ffffffffffff
The following script should work:
macFile = "./macFile.txt"
macTable = {}
if macFile then
local hFile = io.open(macFile, "r")
if hFile then
for line in hFile:lines() do
local _,_, sMac = line:find("^(%x+)")
if sMac then
print("Mac address matched: "..sMac)
table.insert(macTable, sMac)
end
end
hFile:close()
end
end
local output = {}
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
for k,v in ipairs(macTable) do
if s == v then
print("Matched macTable address: "..v)
table.insert(output, v)
end
end
I am just leaving from work and can't have a deeper look at your problem right now, but the next two lines seem quite odd.
t = "00:00:00:00:00:00"
s = string.gsub( t, ":", "")
Basically you are removing all the ':' characters in the string t. Afterwards you end up with s being "000000000000". This is probably not what you want?