Is there an alternate way in perl6 to get file attribute details like size, access_time, modified_time.. etc. without having to invoke native call?
As per the doc it is "unlikely to be implemented as a built in as its POSIX specific".
What workaround options are available excluding the system call to stat?
Any ideas or pointers are much appreciated.
Thanks.
See the IO::Path doc.
For example:
say 'foo'.IO.s; # 3 if 'foo' is an existing file of size 3 bytes
.IO on a string creates an IO::Path object corresponding to the filesystem entry corresponding to the path given by the string.
See examples of using junctions to get multiple attributes at the same time at the doc on ACCEPTS.
I'm not sure if the following is too much. Ignore it if it is. Hopefully it's helpful.
You can discover/explore some of what's available in Perl 6 via its HOW objects (aka Higher Order Workings objects, How Objects Work objects, metaobjects -- whatever you want to call them) which know HOW objects of a particular type work.
say IO::Path.^methods
displays:
(BUILD new is-absolute is-relative parts volume dirname basename extension
Numeric sibling succ pred open watch absolute relative cleanup resolve
parent child add chdir rename copy move chmod unlink symlink link mkdir
rmdir dir slurp spurt lines comb split words e d f s l r w rw x rwx z
modified accessed changed mode ACCEPTS Str gist perl IO SPEC CWD path BUILDALL)
Those are some of the methods available on an IO::Path object.
(You can get more or less with adverbs, eg. say IO::Path.^methods(:all), but the default display aims at giving you the ones you're likely most interested in. The up arrow (^) means the method call (.methods) is not sent to the invocant but rather is sent "upwards", up to its HOW object as explained above.)
Here's an example of applying some of them one at a time:
spurt 'foo', 'bar'; # write a three letter string to a file called 'foo'.
for <e d f s l r w rw x rwx z modified accessed changed mode>
-> $method { say 'foo'.IO."$method"() }
The second line does a for loop over the methods listed by their string names in the <...> construct. To call a method on an invocant given it's name in a variable $qux, write ."$qux"(...).
While looking for an answer to this question in 2021, there is the File::Stat module. It provides some additional stat(2) information such as UID, GID and mode.
#!/usr/bin/env raku
use File::Stat <stat>;
say File::Stat.new(path => $?FILE).mode.base(8);
say stat($?FILE).uid;
say stat($?FILE).gid;
Related
I tried to import System.Directory in my Frege program (In Eclipse) in order to use functions as getDirectoryContent, etc., and it writes me this error :
Could not import module frege.system.Directory (java.lang.ClassNotFoundException: frege.system.Directory)
What do I have to do ?
It is because the module frege.system.Directory doesn't exist in Frege. A good way to find out about a module is to use Hoogle for Frege at this URL: http://hoogle.haskell.org:8081. If we search for that module there, we can see that it doesn't list any module as opposed to, say, if you search for frege.data.List, we would see the module in the result.
Now for the functions you need like getDirectoryContent, if you look at the search result for frege.system.Directory, the first result is about processes and the third and fourth results are about jars and zip files. If you click on the second result, it would open the module frege.java.IO and you can see some relevant functions that might be useful for you (list for example). However the Haskell module you are trying to find is not yet ported to Frege but it should, of course, be possible to port that module backed by native Java implementations.
Update for OP's comment
Here is a simple snippet to return the files under a given directory:
ls :: String -> IO [String]
ls dir = do
contents <- File.new dir >>= _.list
maybe (return []) (JArray.fold (flip (:)) []) contents
Regarding createTempFile, the following works for me:
frege> File.createTempFile "test.txt"
String -> STMutable RealWorld File
How can I ensure that all modules (and ideally also all other files that have been loaded or included) are up-to-date? When issuing use_module(mymodule), SICStus compares the modification date of the file mymodule.pl and reloads it, if newer. Also include-ed files will trigger a recompilation. But it does not recheck all modules used by mymodule.
Brief, how can I get similar functionality as SWI offers it with make/0?
There is nothing in SICStus Prolog that provides that kind of functionality.
A big problem is that current Prologs are too dynamic for something like make/0 to work reliably except for very simple cases. With features like term expansion, goals executed during load (including file loading goals, which is common), etc., it is not possible to know how to reliably re-load files. I have not looked closely at it, but presumably make/0 in SWI Prolog has the same problem.
I usually just re-start the Prolog process and load the "main" file again, i.e. a file that loads everything I need.
PS. I was not able to get code formatting in the comments, so I put it here instead: Example why make/0 needs to guard against 'user' as the File from current_module/2:
| ?- [user].
% compiling user...
| :- module(m,[p/0]). p. end_of_file.
% module m imported into user
% compiled user in module m, 0 msec 752 bytes
yes
| ?- current_module(M, F), F==user.
F = user,
M = m ? ;
no
| ?-
So far, I have lived with several hacks:
Up to 0.7 – pre-module times
SICStus always had ensure_loaded/1 of Quintus origin, which was not only a directive (like in ISO), but was also a command. So I wrote my own make-predicate simply enumerating all files:
l :-
ensure_loaded([f1,f2,f3]).
Upon issuing l., only those files that were modified in the meantime were reloaded.
Probably, I could have written this also like — would I have read the meanual (sic):
l :-
\+ ( source_file(F), \+ ensure_loaded(F) ).
3.0 – modules
With modules things changed a bit. On the one hand, there were those files that were loaded manually into a module, like ensure_loaded(module:[f1,f2,f3]), and then those that were clean modules. It turned out, that there is a way to globally ensure that a module is loaded — without interfering with the actual import lists simply by stating use_module(m1, []) which is again a directive and a command. The point is the empty list which caused the module to be rechecked and reloaded but thanks to the empty list that statement could be made everywhere.
In the meantime, I use the following module:
:- module(make,[make/0]).
make :-
\+ ( current_module(_, F), \+ use_module(F, []) ).
This works for all "legal" modules — and as long as the interfaces do not change. What I still dislike is its verboseness: For each checked and unmodified module there is one message line. So I get a page full of such messages when I just want to check that everything is up-to-date. Ideally such messages would only show if something new happens.
| ?- make.
% module m2 imported into make
% module m1 imported into make
% module SU_messages imported into make
yes
| ?- make.
% module m2 imported into make
% module m1 imported into make
% module SU_messages imported into make
yes
An improved version takes #PerMildner's remark into account.
Further files can be reloaded, if they are related to exactly one module. In particular, files loading into module user are included like the .sicstusrc. See above link for the full code.
% reload files that are implicitly modules, but that are still simple to reload
\+ (
source_file(F),
F \== user,
\+ current_module(_, F), % not officially declared as a module
setof(M,
P^ExF^ExM^(
source_file(M:P,F),
\+ current_module(M,ExF), % not part of an official module
\+ predicate_property(M:P,multifile),
\+ predicate_property(M:P,imported_from(ExM))
),[M]), % only one module per file, others are too complex
\+ ensure_loaded(M:F)
).
Note that in SWI neither ensure_loaded/1 nor use_module/2 compare file modification dates. So both cannot be used to ensure that the most recent version of a file is loaded.
In D3, suppose I have a file called foo and I want to write the contents of the file out to /var/tmp/bar. The documentation leads me to believe that it should be possible to make D3 write the file to the file system by changing the D pointer into a Q pointer, but I can't figure out how to make this happen.
You can do this at least a few ways.
1) You don't want to change a d-pointer to a q-pointer, you just want to create a q-pointer. In other words there's no need to have a d-pointer first to access the host file system. So your q-pointer called 'bar' will look like this:
Q
/var/tmp/bar
With that you can simply:
copy foo
to: (bar
Note that in this case 'bar' is a host OS folder/directory, not a file. A D3 'file' is a table that has multiple records. That translates to a host OS directory with multiple files.
Options are available on the Copy command to suppress the display of item IDs (keys) as records are copied (see docs).
2) You don't even need a q-pointer:
copy foo
to: (/var/tmp/bar
3) Similarly in code you can use the q-pointer or you can use the direct path:
open 'bar' to f.bar1 ...
open '/var/tmp/bar' to f.bar2 ...
==
The path syntax is using a mechanism called the OSFI (see docs). With this syntax you can specify a driver. The default driver called "unix:" converts attribute marks to the *nix EOL which is a line-feed x0A. If you're on Windows the default is "dos:" which converts attribute marks to CRLF x0D0A. You can force a non-default by preceding the path with the driver. So to create a DOS-format file in Unix/Linux, use dos:/var/tmp/bar. The default drivers also convert between tab and 4-spaces (see docs). Values and subvalues are not converted but a new driver can be created to do so. Use the 'bin:' driver to avoid conversions, so bin:/var/tmp/bar will not convert #am (xFE) to x0A, etc.
If you need more detail I'll be happy to add to this.
I have a Matlab program that does something like this
cd media;
for i =1:files
d(i).r = %some matlab file read command
d(i).process();
end
cd ..;
When I change to my "media" directory I can still access member properties (such as 'r'), but Matlab can't seem to find functions like process(). How is this problem solved? Is there some kind of global function pointer I can call? My current solution is to do 2 loops, but this is somewhat deeply chagrining.
There are two solutions:
don't change directories - instead give the file path the your file read command, e.g.
d(i).r = load(['media' filesep 'yourfilename.mat']);
or
add the directory containing your process() to the MATLAB path:
addpath('C:\YourObjectsFolder');
As mentioned by tdc, you can use
addpath(genpath('C:\YourObjectsFolder'));
if you also want to add all subdirectories to your path.
Jonas already mentioned addpath, but I usually use it in combination with genpath:
addpath(genpath('path_to_folder'));
which also adds all of the subdirectories of 'path_to_folder' as well.
I'm using rsync to make a backup of my server files, and I have two questions:
In the middle of the process I need to stop and start rsync again.
Will rsync start from the point where it stopped or it will restart from the beginning?
In the log files I see "f+++++++++". What does it mean?
e.g.:
2010/12/21 08:28:37 [4537] >f.st...... iddd/logs/website-production-access_log
2010/12/21 08:29:11 [4537] >f.st...... iddd/web/website/production/shared/log/production.log
2010/12/21 08:29:14 [4537] .d..t...... iddd/web/website/production/shared/sessions/
2010/12/21 08:29:14 [4537] >f+++++++++ iddd/web/website/production/shared/sessions/ruby_sess.017a771cc19b18cd
2010/12/21 08:29:14 [4537] >f+++++++++ iddd/web/website/production/shared/sessions/ruby_sess.01eade9d317ca79a
Let's take a look at how rsync works and better understand the cryptic result lines:
1 - A huge advantage of rsync is that after an interruption the next time it continues smoothly.
The next rsync invocation will not transfer the files again, that it had already transferred, if they were not changed in the meantime. But it will start checking all the files again from the beginning to find out, as it is not aware that it had been interrupted.
2 - Each character is a code that can be translated if you read the section for -i, --itemize-changes in man rsync
Decoding your example log file from the question:
>f.st......
> - the item is received
f - it is a regular file
s - the file size is different
t - the time stamp is different
.d..t......
. - the item is not being updated (though it might have attributes
that are being modified)
d - it is a directory
t - the time stamp is different
>f+++++++++
> - the item is received
f - a regular file
+++++++++ - this is a newly created item
The relevant part of the rsync man page:
-i, --itemize-changes
Requests a simple itemized list of the changes that are being made to
each file, including attribute changes. This is exactly the same as
specifying --out-format='%i %n%L'. If you repeat the option, unchanged
files will also be output, but only if the receiving rsync is at least
version 2.6.7 (you can use -vv with older versions of rsync, but that
also turns on the output of other verbose messages).
The "%i" escape has a cryptic output that is 11 letters long. The
general format is like the string YXcstpoguax, where Y is replaced by
the type of update being done, X is replaced by the file-type, and the
other letters represent attributes that may be output if they are
being modified.
The update types that replace the Y are as follows:
A < means that a file is being transferred to the remote host (sent).
A > means that a file is being transferred to the local host (received).
A c means that a local change/creation is occurring for the item (such as the creation of a directory or the changing of a symlink,
etc.).
A h means that the item is a hard link to another item (requires --hard-links).
A . means that the item is not being updated (though it might have attributes that are being modified).
A * means that the rest of the itemized-output area contains a message (e.g. "deleting").
The file-types that replace the X are: f for a file, a d for a
directory, an L for a symlink, a D for a device, and a S for a
special file (e.g. named sockets and fifos).
The other letters in the string above are the actual letters that will
be output if the associated attribute for the item is being updated or
a "." for no change. Three exceptions to this are: (1) a newly created
item replaces each letter with a "+", (2) an identical item replaces
the dots with spaces, and (3) an unknown attribute replaces each
letter with a "?" (this can happen when talking to an older rsync).
The attribute that is associated with each letter is as follows:
A c means either that a regular file has a different checksum (requires --checksum) or that a symlink, device, or special file has a
changed value. Note that if you are sending files to an rsync prior to
3.0.1, this change flag will be present only for checksum-differing regular files.
A s means the size of a regular file is different and will be updated by the file transfer.
A t means the modification time is different and is being updated to the sender’s value (requires --times). An alternate value of T
means that the modification time will be set to the transfer time,
which happens when a file/symlink/device is updated without --times
and when a symlink is changed and the receiver can’t set its time.
(Note: when using an rsync 3.0.0 client, you might see the s flag
combined with t instead of the proper T flag for this time-setting
failure.)
A p means the permissions are different and are being updated to the sender’s value (requires --perms).
An o means the owner is different and is being updated to the sender’s value (requires --owner and super-user privileges).
A g means the group is different and is being updated to the sender’s value (requires --group and the authority to set the group).
The u slot is reserved for future use.
The a means that the ACL information changed.
The x means that the extended attribute information changed.
One other output is possible: when deleting files, the "%i" will
output the string "*deleting" for each item that is being removed
(assuming that you are talking to a recent enough rsync that it logs
deletions instead of outputting them as a verbose message).
Some time back, I needed to understand the rsync output for a script that I was writing. During the process of writing that script I googled around and came to what #mit had written above. I used that information, as well as documentation from other sources, to create my own primer on the bit flags and how to get rsync to output bit flags for all actions (it does not do this by default).
I am posting that information here in hopes that it helps others who (like me) stumble up on this page via search and need a better explanation of rsync.
With the combination of the --itemize-changes flag and the -vvv flag, rsync gives us detailed output of all file system changes that were identified in the source directory when compared to the target directory. The bit flags produced by rsync can then be decoded to determine what changed. To decode each bit's meaning, use the following table.
Explanation of each bit position and value in rsync's output:
YXcstpoguax path/to/file
|||||||||||
||||||||||╰- x: The extended attribute information changed
|||||||||╰-- a: The ACL information changed
||||||||╰--- u: The u slot is reserved for future use
|||||||╰---- g: Group is different
||||||╰----- o: Owner is different
|||||╰------ p: Permission are different
||||╰------- t: Modification time is different
|||╰-------- s: Size is different
||╰--------- c: Different checksum (for regular files), or
|| changed value (for symlinks, devices, and special files)
|╰---------- the file type:
| f: for a file,
| d: for a directory,
| L: for a symlink,
| D: for a device,
| S: for a special file (e.g. named sockets and fifos)
╰----------- the type of update being done::
<: file is being transferred to the remote host (sent)
>: file is being transferred to the local host (received)
c: local change/creation for the item, such as:
- the creation of a directory
- the changing of a symlink,
- etc.
h: the item is a hard link to another item (requires
--hard-links).
.: the item is not being updated (though it might have
attributes that are being modified)
*: means that the rest of the itemized-output area contains
a message (e.g. "deleting")
Some example output from rsync for various scenarios:
>f+++++++++ some/dir/new-file.txt
.f....og..x some/dir/existing-file-with-changed-owner-and-group.txt
.f........x some/dir/existing-file-with-changed-unnamed-attribute.txt
>f...p....x some/dir/existing-file-with-changed-permissions.txt
>f..t..g..x some/dir/existing-file-with-changed-time-and-group.txt
>f.s......x some/dir/existing-file-with-changed-size.txt
>f.st.....x some/dir/existing-file-with-changed-size-and-time-stamp.txt
cd+++++++++ some/dir/new-directory/
.d....og... some/dir/existing-directory-with-changed-owner-and-group/
.d..t...... some/dir/existing-directory-with-different-time-stamp/
Capturing rsync's output (focused on the bit flags):
In my experimentation, both the --itemize-changes flag and the -vvv flag are needed to get rsync to output an entry for all file system changes. Without the triple verbose (-vvv) flag, I was not seeing directory, link and device changes listed. It is worth experimenting with your version of rsync to make sure that it is observing and noting all that you expected.
One handy use of this technique is to add the --dry-run flag to the command and collect the change list, as determined by rsync, into a variable (without making any changes) so you can do some processing on the list yourself. Something like the following would capture the output in a variable:
file_system_changes=$(rsync --archive --acls --xattrs \
--checksum --dry-run \
--itemize-changes -vvv \
"/some/source-path/" \
"/some/destination-path/" \
| grep -E '^(\.|>|<|c|h|\*).......... .')
In the example above, the (stdout) output from rsync is redirected to grep (via stdin) so we can isolate only the lines that contain bit flags.
Processing the captured output:
The contents of the variable can then be logged for later use or immediately iterated over for items of interest. I use this exact tactic in the script I wrote during researching more about rsync. You can look at the script (https://github.com/jmmitchell/movestough) for examples of post-processing the captured output to isolate new files, duplicate files (same name, same contents), file collisions (same name, different contents), as well as the changes in subdirectory structures.
1.) It will "restart the sync", but it will not transfer files that are the same size and timestamp etc. It first builds up a list of files to transfer and during this stage it will see that it has already transferred some files and will skip them. You should tell rsync to preserve the timestamps etc. (e.g. using rsync -a ...)
While rsync is transferring a file, it will call it something like .filename.XYZABC instead of filename. Then when it has finished transferring that file it will rename it. So, if you kill rsync while it is transferring a large file, you will have to use the --partial option to continue the transfer instead of starting from scratch.
2.) I don't know what that is. Can you paste some examples?
EDIT: As per http://ubuntuforums.org/showthread.php?t=1342171 those codes are defined in the rsync man page in section for the the -i, --itemize-changes option.
Fixed part if my answer based on Joao's