Detect new content within a file, extract only the new things - nio2

it sounds like a stupid question and maybe I'm just blocked by whatever ... Using the new NIO2 of Java I can perfectly track the occurrance of new files, I can parse them and send the content so somewhere. But I have no glue how I can easily track the update and read the new stuff only ....
As always: any hint appreciated

You track the update by watching the parent directory for ENTRY_MODIFY.
Now for the bad news: Nothing in the Java in API tells you what is new.
To learn about the new stuff you can use several methods e.g.
a) keep a copy of interesting files and send the diff
b) use a wrapping filesystem that hooks into writing and sends from there

Related

LibGit2Sharp: how to Task-Async

Hi I have begun to use the package for some very simple tasks, mainly cloning a Git-Wiki repo and subsequently pulling the changes from the server when needed.
Now I can not see any methods corresponding to the Task-Async (TAP) pattern. Also in the documentation I could not find anything concerning.
Could you please give me some direction how to wrap the LibGit2Sharp methods into a TAP construct? Link to documentation (if I missed something) or just telling me which callback to hook up to the TaskCompletionSource object would be nice.
It also doesn't really help that I am a newbie with Git, and normally I only do basic branching, merging, pushing with it.
For cloning I use:
Repository.Clone(#"https://MyName#bitbucket.org/MyRepo/MyProject.git/wiki", "repo");
For pulling I use:
using (var repo = new Repository("repo"))
{
// Credential information to fetch
LibGit2Sharp.PullOptions options = new LibGit2Sharp.PullOptions();
options.FetchOptions = new FetchOptions();
var signature = new LibGit2Sharp.Signature(new Identity("myname", "mymail#google.com"), DateTimeOffset.Now);
Commands.Pull(repo, signature, options);
}
Thanks in advance
First of all, you should never try sync over async or async over sync. See this article.
If you're thinking of using Task.Run, don't. That will just trade on thread pool thread for another with the added cost of 2 context switches.
But you should reconsider your whole approach to this. You don't need to clone the repository just to get the contents of a file. Each version of a file, has a unique URL. You can even get the URL of a file for a specific branch.

Correct terminology for Dead Man Switch

I've implemented a Dead Man Switch this way:
A script can be fired by an event. When the script starts it looks for a specific object on an S3 bucket. If that object can't be found for any reason (be it network issues to access the bucket, lack of permissions, the object was removed or any other reason) then the script will abort before doing any other actions.
I suppose this is a classic Dead Man Switch.
The idea is to let us remove this object in case we need to stop the script in an emergency.
My question is about terminology - I also supply a script to our team to create or remove that S3 object. I want it to be clear which actions means what (remove object - stop script from doing anything, create object - let the script continue with its work). I used "removed" and "reinstate" and was told this is too ambiguous. I now contemplate about "pushed" and "enabled" but this too sounds too vague. I'm also thinking about "pulled" (object removed) vs. "rearmed" (object created).
It's important that the terminology will be clear since if this script is used then this is expected to happen during emergency, so we want to minimize confusion as much as possible.
I suppose the problem is the inherent "double negative".
So far I didn't find any common name used to describe these actions. Wikipedia and other places describe what the switch is, but not actions of enabling or disabling it.
Any ideas?
I would say that the system would be:
Enabled if the object is present
Disabled if the object is absent
Placing the object would enable the script.
Removing (deleting) the object would disable the script.
You could even call the file enable-xxx-script to make it more obvious.

Yii load system params from Database using Active Record

I am interested in loading some system params into the Yii::app()->params array from the database using a CActiveRecord extension called SiteSetting.
Unfortunately I couldn't find much advice online for this, but believe I can place a method in SiteSetting called loadSiteSettingsToAppParams and add the setting...
'onBeginRequest'=>array('SiteSetting', 'loadSiteSettingsToAppParams')
...to the config.
I would like to know if I can only add this onBeginRequest to the Yii::app() somewhere within the SiteSetting class (to keep my code modular) and whether this is a sensible approach.
Thanks in advance.
Just re-read your question now and I'd try to provide answers.
To the question "I would like to know if I can only add this onBeginRequest to the Yii::app() somewhere within the SiteSetting class (to keep my code modular)": the answer is, You're not restricted to just a Class. You could (theoretically) place it anywhere within your application and also in the config.php file.
As to whether it's a sensible approach, it depends on the time it would take to request those settings from the database and whether you're prepared to add that time to your HttpRequest response time. The onBeforeRequest is fired before every HttpRequest and if the loadSiteSettingsToAppParams method consumes lots of time, you're adding that time to your HttpRequest response time.
I'd advise that you fetch those settings once after login and then update them only when they change (the settings are updated). This way, you could place the call to loadSiteSettingsToAppParams in the UserIdentity class and call it after a successful login.
That's just how I'd go about doing this though.
Hope I helped.
The easy & nice way to accomplish this by using a comoponent like SettingComoponent and place in the components directory protected/components then pre load this component in the preload section like this preload => array('log', 'setting', ...). That's it and now you can call this component anywhere you want like Yii:app()->setting->whatever.
I hope this is answer can be useful for you.

Best multithreading approach in Objective C?

I'm developing an iPad-app and I'm currently struggling with finding the best approach to multithreading. Let me illustrate this with a simplified example:
I have a view with 2 subviews, a directory picker and a gallery with thumbnails of all the images in the selected directory. Since 'downloading' and generating these thumbnails can take quite a while I need multithreading so the interaction and updating of the view doesn't get blocked.
This is what I already tried:
[self performSelectorInBackground:#selector(displayThumbnails:) withObject:currentFolder];
This worked fine because the users interactions didn't get blocked, however it miserably fails when the user taps on another folder while the first folder is still loading. Two threads are trying to access the same view and variables which results in messing up each others proper execution. When the users taps another folder, the displayThumbnails of the currently loading folder should get aborted. I didn't find any way to do this..
NSThreads
I tried this but struggled with almost the same problems as with the first method, I didn't find a (easy) way to cancel the ongoing method. (Yes, I know about [aThread cancel] but didn't find a way to 'resume' the thread). Maybe I should subclass NSThread and implement my own isRunning etc methods? But isn't there any better way or a third (or even fourth and fifth) option I'm overlooking?
I think this is a fairly simple example and I think there is perhaps a better solution without subclassing NSThread. So, what would you do? Your opinions please!
NSOperationQueue should work well for this task.
Another option would be plain GCD, however, if you've never worked with it, NSOperationQueue is probably the better choice since it pretty much automatically guides you to implementing things "the right way", has obvious ways for cancellation, etc.
You want to use Concurrent NSOperations to download and process images in the background. These would be managed by an NSOperationsQueue. Essentially these operations would be configured to fetch one image per operation, process it, save it in the file system, then message back to the main app in the main thread that the image is available.
There are several projects on github that you can look at that show how to do this - just search github using "Concurrent" or "NSOperation".
iOS has a really nice facility for doing background work. Grand Central Dispatch (GCD) and Blocks, but those don't let you have an object using delegate callbacks - thus NSOperation.
So you need to read up on blocks, GCD, and then look at some open source Concurrent NSOperations code. Using Concurrent NSOperations is not as simple as using blocks.
If I had this problem, I would probably go for an approach like this:
a single thread that will load the images, and causes the main thread to display results (I'm not a big fan of having thread mess around with GUI objects)
when a new directory is requested... well, it depends on how you want to manage things. Basically, a standard queue construct (condition variable and array) could be used for the main thread to tell the thread that "this directory will be needed" by passing it the path name; the thread will check the queue even when it's loading images (like after every image or so), and switch to the new directory whenever one shows up
you could make a directory-reader object that keeps all the state, and store this indexed by the path into a dictionary. When a new directory is requested, check that dictionary first, and only create a new object if there's none for this directory. That way, partially loaded directories would stick around until they are needed again, and can continue to load instead of having to start from scratch.
Pseudocode for the thread:
while (forever)
new element = nil
if we have an active directory loader
tell directory loader to load one image
if false then make directory loader inactive
lock queue condition
if queue has elements
new element = retrieve LAST element (we aren't interested in the others)
empty queue
unlock with status "empty"
else
unlock queue
else
lock queue on condition "has elements"
new element = retrieve last element
empty queue
unlock with status "empty"
if new element != nil
if directory loader for new path does not exist
setup new directory loader for new path
store in dictionary
make it the "active" one
else
make the current one the "active"
As for the directory loader, it might look something like this:
read one image:
if there are still images to read:
read, process and store one
return true
else
performSelectorOnMainThread with an "update GUI" method and the image list as parameter
return false;
This is just a quick sketch; there's some code duplication in the thread, and the way I wrote it will only update the GUI after all images have been read, instead of making them appear as we read them. You'll have to copy the current image list, or add synchronization if you want to do that.

Stopping invalid file type or file name submissions in coldfusion

So, I'm having this lovely issue where people like to submit invalid file types or funky named files... (like.. hey_i_like_"quotes".docx) Sometimes they will even try to upload a .html link...
How should I check for something like this? It seems to create an error every time someone submits a poorly named item.
Should I create a cfscript that checks it before submission? Or is there an easier way?
If it was before submission it would be javascript not cfscript. Javascript can always be got round, so I'd say you'd be better doing it server-side with ColdFusion. Personally I'd just wrap the whole thing in a try/catch (you should do this anyway as a matter of course with all file upload type things), and throw an error back at them if their filename is no good.
When you say submit are you using cffile to allow your users to upload file.
If so, use the attribute "accept" with a try and catch around. for example....
<cftry>
<cffile action = "upload"
fileField = "FileContents"
destination = "c:\files\upload\"
accept="image/jpg, application/msword"
>
<cfcatch type="Any" >
<p>sorry we could not upload your file!</p>
</cfcatch>
</cftry>
I personally would not use "just" JavaScript as this could be disabled and you are back in the same boat.
Hope this helps.
On the server, as part of validation, use reFindNoCase() along with an appropriate regex to check for a properly formatted file path. You can find lots of example regex expressions for a file path on the internet, such at this one. Hope that helps.
As #Duncan pointed out, a client-side validation would most likely be in JavaScript. Personally, if I had time/resources, I would do this as a convenience for the end user. If they upload an enormous PDF when a DOCX is required by the system, it would be annoying for them not to receive a message until the upload is complete.
As far as filenames go, it seems to me that the simplest solution (and one I've used in the past) is to assume all filenames are bad, and rename them. There are several ways to do this. If you need to preserve the original filename, I would just use urlEncodedFormat() ot clean the filename into something that is web-friendly. If you need to preserve all versions, you can append a date/time stamp, so bob.xocx becomes bob_201104051129.docx or somesuch. If you must keep the original filename without any changes, I would recommend seting up a DB table as a pinter system, keeping the original name, timestamp, and other metadata there and referring to the file by renaming it to the ID.
But urlEncodedFormat() is probably enough for what you've outlined.
For user experience it's best to do it client-side but it's not bad at all to double check server side too.
For the client side part, I recommend using the jQuery validation plugin, easy to use.