my app checks a website for some files with simple NSURLConnection Method. Now I want to recognize if one of the files has changed without downloading the file and compare it.
I thought about md5 checksums but how can I do this without wasting traffic downloading the whole file.
Do you have any ideas for this?
Check the timestamp on the file. That should be easier then using md5 checksums. I don't know how your app or or server API is implemented but the idea is pretty straightforward:
On the server create an API that allows you to query when a file was last modified (keeping track of the modification timestamps should already be handled by the OS on the server)
When you download the file on your client also store the timestamp (i.e. when the server thinks the file was last modified).
When checking whether to update a file, first ask the server timestamp for the file and compare it with the one in your client app - if the server timestamp is more recent than the one on your client download the new file, otherwise do nothing.
Related
I have a website that has the old "list files" style of doing things, and I want to perform a hash on a file there before downloading it to the user's local system. I know how to hash a local file, but it seems there's not a lot of info as to whether or not I can do this without downloading the online file. My logic is, if the user already has the same file, why waste time downloading it? So, is it possible to do this?
After further contemplation I decided that the date modified comparison is actually the behavior that I want. If a client were to modify a file on accident, there is now an option to correct it. If they modify it on purpose, I certainly don't want to wipe out their work.
I have been given a program that uploades pdf files to an ftp server, which is something I never did. I've been asked what the behavior regarding attempting to upload a duplicate filename is. It apparently doesnt check for duplicate filenames manually, but the comand that uploads the file is My.Computer.Network.UploadFile and I can't find what happens when attempting to upload a duplicate file anywhere, does it throw an exception or overwrites the file?
It looks like My.Computer.Network.UploadFile is a wrapper around WebClient.UploadFile, and the documentation for that states:
This method uses the STOR command to upload an FTP resource.
In the FTP RFC 959 it says (I highlighted the relevant part):
STORE (STOR)
This command causes the server-DTP to accept the data
transferred via the data connection and to store the data as
a file at the server site. If the file specified in the
pathname exists at the server site, then its contents shall
be replaced by the data being transferred. A new file is
created at the server site if the file specified in the
pathname does not already exist.
So, if everything is following standards (and that part of RFC 959 hasn't been replaced, I didn't dig further!), then it should replace the existing file. However, it is possible for the server to deny overwriting of existing files, so the behavior is not guaranteed.
Of course, the best thing to do would be to try it out in your environment and see what it does.
Im using a web service to populate a tableview, but the data in my web database changes very rarely, and i don't need to always download the same data.
I need to storage the data in the device, and verify if the data on the device is different than the one on the web.
What is the best practice to do that? Parse a JSON object, containing the last version and comparing to the local version and then parsing the data if its needed ?
Is there a way to automate the last version?
If you have control over the web APIs then have a separate api that returns a timestamp for the last web data change and check this. If the date is not newer than the date of last download use the local data. imho.
I personally used local storage for the same situation on iOS and it worked great.
You just need to provide an URL which responds with the last version of the data (maybe a date) and then locally in JavaScript you first check if version online is newer than your local stored one, if it is you redownload data and place it into localstorage, otherwise you just keep old data (and don't download anything at all).
Of course everything must be done through Ajax.
Does the NSFileManager method contentsEqualAtPath:andPath: create a dynamic checksum to compare two files, does it open the file header and compare file header details or does it use some other method for comparing?
I have a list of 200,000 or so files to compare where the local files are to be compared with the files on a remote server volume. The local files would have been copied from the remote server volume at some point in the past, and I will be walking the list of files to compare each and then copying over any newer files from the remote server volume to the local machine (overwriting any existing). There is no guarantee that the remote server files were created by the local user (and more than likely they would not have been).
As the files are small (approx. 4K in size) a complex file compare operation might take almost as long as a copy operation.
This operation could (conceivably but not likely) happen multiple times in a user session so I need to make sure that I am using the most efficient method for checking.
The operation itself will run on a separate thread so I don't have issues of tying up the user while the operation completes.
I've started the implementation to test this but was interested to see if anyone else has had any experience comparing thousands of files quickly in order to determine which files need updating if a newer one exists. And if you have, do you have any pointers or pitfalls to avoid?
Any advice much appreciated.
Update
Thinking about this some more, it might be more beneficial to keep a file that tracks the last updated timestamp of any changed images and keep a local file that does the same and just compare those two documents... Will update more as I progress.
It looks to me that for directories only filenames (and filenames of subdirectories) are compared. It only compares file content if you explicitly pass file paths to the method.
On my site a user may upload a file (pic, zip, audio, video, whatever). He then may decide to replace it with a newer revision. This user may upload a file, make a post then decide to put up a new revision replacing the old (lets say its a large zip or tar.gz file). Theres a good chance people may be downloading it if he sent out an email or even im for the home user.
Problem. I need to replace the file and people may be downloading and it may be some minutes before it is deleted. I dont want my code to stall until i cant delete or check every second to see if its unused (especially bad if another user can start and he takes long creating a cycle).
How do i delete the file while users are downloading the file? i dont care if they stop i just care that the file can be replaced and new downloads are the new revision.
What about referencing the files indirectly?
A mapping script, maps a virtual file entry from your site to a real file . If the user wants to upload a new revision of his file you just update the mapping, not the real file.
You can install a daily task that scans all files and deletes all files without a mapping and without open connections.
lajuette's answer is right, the easiest solution is to work around the file locking altogether:
When a user uploads file foo.zip, internally store it as foo-v1.zip.
Create a mapping file somewhere (database, code, whatever) that maps foo.zip to foo-v1.zip.
Rather than exposing a direct link to the file, expose a link to a service that gets the file: mysite.com/Download?foo.zip or something. This service uses the mapping to determine which version of the file to send to the client.
When a new version is uploaded, create foo-v2.zip and update the mapping file.
It wouldn't be that hard to write a scheduled task that cleans up old, un-mapped files.
If your oppose to a database and If the filenames are in a fix format (such as user/id.ext) you could append the id with a revision number and enumerate the folder using a pattern (user/id-*) and use the latest revision.