I'm working with a GPS module that is transferring data to my mac over a serial RS232-to-USB interface. I've written a objC program that takes the raw data and converts it into meaningful information.
Using a program called goSerial, I'm able to log all incoming data into a text file. I have been able to make my program read the text file and process data line by line.
I would like this procedure to happen in real time i.e. as soon as the data is received, it gets logged into the text file and my program reads it. The first part of this happens automatically that is the text file is being constantly appended (when not open). Is it possible to monitor a text file for appended data and only read new lines? Also, will doing this affect the ability of new incoming data to be saved?
Thanks!!!
(Also, if anyone knows how I may send serial data directly to Xcode, please let me know!)
I'm not sure how the serial-to-USB affects things but traditionally, unix accesses serial devices using the Device-File Mechanism which treats the input from the device as a file to be read. You would use NSFileHandle to read the file from Cocoa/Foundation. You probably want to checkout the IORegistryExplorer app to see how your device shows up.
You can use a kqueue (perhaps with a wrapper such as UKKQueue) to watch the file for changes.
You should be able to create a unix domain socket, which you can then have your goSerial application open (as it looks like a normal file on the fs)
And then read from the other end, linewise in your application. This is probably the easiest way, or alternately have a look at the source of tail in GNU's coreutils, specifically it's -f function (although thinking more you'll probably want to look at how the FreeBSD implementation works, as I believe that the GNU version uses some linux specific callback)
Related
I have a text file. Now I have changed its file type from .txt to .abc. My VB.NET program loads the text into textboxes from that file. After changing the file type, however, other apps like NotePad and Word are able to open and read my .abc file.
Is there any way that only my application will be able to open/read from the file and no other app would be able to do so? What I mean is, suppose I have a PhotoShop document .psd file, no other app, rather that photoshop itself, can open it. How do I make my file unreadable by other apps?
There is no way to prevent an app that you don't develop from opening any file. The extensions are just there for helping us humans, and maybe a bit for the computer to know the default app you select for an extension.
Like you said, a .txt file can be opened by many many apps. You can open a .txt file with Notepad, Firefox, VSCode, and many others.
Same way, a .psd file can be opened by many many apps. You can open that .psd file with Photoshop, but also Notepad, Firefox, and VSCode, and probably the same apps as above.
The difference is which apps can read and understand the file.
In order to make a file not understandable by other apps, you need to make it into a format that cannot recognize, because you planned it "in secret".
Like Visual Vincent said above, you could encrypt the file in a way, or you can have a binary file, that basically only your app knows know to understand.
Since you dont own the app you want the file to be understood by, then you either have to accept that it can be opened by any app that can open files, or you can try to encrypt the file outside the app, or like zipping it with a password, and then decrypting or unzipping when you want to use it.
Firstly, any file can be read unless it is still open by a particular process or service. Even PhotoShop files can be 'read' by NotePad - try it!
So, an attempt at my first answer...
You can try a couple of methods to prevent opening the file, for instance, applying a file lock. As an example, SQL Server .mdf files are locked by the SQL Server service. This happens because the files are maintained in an open state, however; your application would have to remain running to keep these files open. Technically, though, the files can still be copied.
Another way is to set the hidden attribute for the file. This hides the file from the less savvy users, but it will be displayed if the user show's hidden files.
And my second answer: You refer to the format of files by saying only PhotoShop can read or write its own files (not true, but I know what you're saying).
The format of the file must be decided by yourself. You must determine how you are going to store the data that you output from your application. It looks like you have been attempting to write your application data into a text file. Perhaps you should try writing to binary files instead. Binary files, while not encrypted, as suggested by Visual Vincent in the comments to your question, still provide a more tailored approach to storing your data.
Binary files write raw binary data instead of humanised text. For instance, if you write an integer to the file it will appear as a string of four bytes, not your usual 123456789 textual format.
So, you really need to clarify what data you want to write to the file, decide on a set structure to your file (as you also have to be able to read it back in to your application) and then be able to write the information.
My app is monitoring a "hot" folder somewhere on the local filesystem for newly added files to push to a network location. I'm running into a problem when very large files are being written into the hot folder: the file system event notifying me of changes in the hot folder will fire well before the file completes writing. When my app tries to upload the file, it mis-reads the file size as the current number of copied bytes, not the eventual total number of bytes.
Things I've tried:
NSURL getResourceValue:forKey:error: to read NSURLAllocatedFileSizeKey (same value as NSURLFileSizeKey while the file is being written).
NSFileManager attributesOfItemAtPath:error: to look at NSFileBusy (always NO).
I can't seem to find any mechanism short of repeatedly polling a file for its size to determine if the file is finished copying and can be uploaded.
There aren't great ways to do this.
If you can be certain that the writer is using NSFileCoordinator, then you can also use that to coordinate your access to the file.
Likewise, if you're sure that the writer has opted in to advisory locking, you could try to open the file for shared access by calling open() with the O_SHLOCK and O_NONBLOCK flags. If you succeed, then there are no other descriptors open for exclusive access. You can either use the file descriptor you've got or close it and then use some other API to access the file.
However, if you can't be sure of any of those, then your best bet may be to set a timer to repeatedly check the file's metadata (size, date modified, etc.). Only when you see that it has stopped changing over a reasonable time interval (2 seconds, maybe) would you attempt to access it (and cancel the timer).
You might want to do all three. Wait for the file's metadata to settle down, then use a NSFileCoordinator to read from the file. When it calls your reader block, use open() with O_SHLOCK | O_NONBLOCK to make sure there are no other processes which have exclusive access to it.
You need some form of coordinated file locking.
fcntl() and flock() are common functions for this.
Read up on it first.
Then see what options you have.
If you can control the code base of those other processes, all the better.
The problem with really large files is that what's changed or changing inside them is opaque and isn't always at the end.
Good processes should generally be doing atomic writes. (Write to a temp file then swap it out) but if these files are actually databases then you will want to look at using the db's server app for this sort of thing.
If the files are wrappers containing other files then it gets extra messy as those contents might have dependencies on one another to be in a usable state.
I am trying to watch files in a directory to determine when files are opened/accessed. I thought FileSystemWatcher would do the trick using the event Changed.
Problem is that some applications do not create a lock on the file they open/access or change either the date modified or date accessed (even after fsutil behavior set disablelastaccess 0). Notepad for example. Apparently is makes a copy of the file in memory and plays with it there until you save it. Nor does it update the Date Accessed.
How can I monitor a directory of files and be notified when a file is simply opened/accessed by any program (e.g. Notepad)? Files may be opened from another computer, not necessarily on the computer running the "watcher".
I found lots of similar questions but did not see one focusing on file "access".
This is quite normal. Updating an existing file is quite dangerous since it can cause irretrievable data loss. A disk error (like disk full) while writing is very bad news. The common algorithm used:
rename the original file
write a new file using the original name
no error: delete the renamed file
error: delete the new file, rename original file back
Clearly this doesn't cause a Changed event to be raised, no file was changed.
Sorry, I didn't read the question well enough. There is no notification whatsoever for an app just opening a file for reading. FSW can only detect changes to the file system. There is no ready alternative either, this requires a custom file system filter driver that snoops on driver requests. Like the kind that SysInternals' ProcMon utility uses. I'm not aware of such a driver ready for use in a C# program, you can't write them in C# either. This just isn't a common requirement.
Suppose a user selects a file in a dialogue box, and the app then opens the file for reading, etc. Users can open "incorrect" files--they can select a binary file, for example, even if the file they're supposed to be selecting is a text file.
I recognize that sometimes improper file types generate exceptions, which can be handled. But sometimes the files don't create exceptions; instead, they just cause the application to work improperly.
What's the standard way to code for these kinds of situations?
Put a unique identifier into the file (usually the first line or some tag)
Restrict the file extension
Do a check on the file whether it's OK
Use 1. if possible or use both 2. and 3.
A lot of operating systems help you out with this by providing filesystem APIs that are at least somewhat file-type-aware (in Cocoa for Mac OS X, there's a setAllowedFileTypes: method on NSOpenPanel, for example). Aside from that, you should make sure to define your file format in a way that's easy for you to identify when your program opens a file. A few well-known bytes at the start of your file is probably enough to protect you from most random-file problems.
I've got VBA code that generates a text file with some pretty basic information included. I then upload that file via FTP.
I got a message from the server admin of the IBM mainframe today that my file was in variable blocking (VB) format and their job process uses a fixed blocking (FB) up to a max size of 256.
How is this done? During the file creation? 3rd party tool?
B
You can simply convert the VB file into FB in mainframe before running the actual process.VB to FB conversion JCL is a small JCL step to do your conversion
You can use Locsite to set the record format on the host dataset(File).
You can find all the list of FTP sub commands in the below user guide
IP User’s Guide and Commands SC31-8780-05
Sorry all, I have a feeling I didn't explain this correctly, because I do now have an answer which is rather simple. These 2 commands seemed to have setup the environment correctly for the file to be fb and not vb.
ftp> quote site lr=94
ftp> quote site rec=fb
If I rightly remember FB is in multiples of the block sizes, that is just how DASD stores the files on disk, it must fit in that multiple block size, which increases speed and throughput on the Mainframe. If the data file is not within the boundary of multiple block sizes (This has nothing to do with the actual size of the data), the DASD system just access files in blocks of 256 bytes...there will be a host of special fields inserted into the data file to describe the blocking and so on...which will get inserted when transferred to the mainframe and that data gets transferred to magnetic tape backups...
There should be a script available on the Mainframe to convert it using JCL (Job Control Language)..ask the Mainframe administrator to do it for you...
By the way it should be noted that the character set you used in your data file, just be aware that the mainframe uses EBCDIC character set...There are plenty of tools out there that can convert from ASCII data to the format to be readable by the mainframe, just something to bear in mind of...If the data gets converted that could impact the file size...Thought it would be worth mentioning and important!
There is a Unix/Linux utility that can convert the data to a fixed block size using the dd utility, although I do not think it would be the right way to do it...
Here's a useful link that will help you in understanding this. And also here on SO a similar user was asking about MVS/TSO data...