I have a nice question here. I'm using Objective C for Mac OS X to create a class which uploads to and downloads from an FTP server. My question is:
Is it possible to read from a file on a FTP server via one stream, while writing to the same file from another stream, not letting the one stream read any further than the other has written?
I already have something in place to prevent the one stream from reading more than it should. All I need to know is can I read from a file on a FTP server while it is being written to?
Thanks.
This is specific to the FTP server and the way it opens files, but in general the answer would be no. Most servers would open the file in exclusive mode when the file is downloaded.
Related
How can I make sure that a file uploaded through SFTP (in a Linux base system) stays locked during the transfer so an automated system will not read it?
Is there an option on the client side? Or server side?
SFTP protocol supports locking since version 5. See the specification.
You didn't specify, what SFTP server are you using. So I'm assuming the most widespread one, the OpenSSH. The OpenSSH supports SFTP version 3 only, so it does not support locking.
Anyway, even if your server supported file locking, most SFTP clients/libraries won't support SFTP version 5. Or even if they do, they won't support the locking feature. Note that the lock is explicit, the client has to request it.
There are some common workarounds for the problem:
As suggested by #user1717259, you can have the client upload a "done" file, once an upload finishes. Make your automated system wait for the "done" file to appear.
You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.
Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after an upload to its final name. Make your automated system ignore the ".filepart" files.
See (my) article Locking files while uploading / Upload to temporary file name for example of implementing this approach.
Also, some SFTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive (courtesy of #fakedad).
A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.
You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, when you download an incomplete file.
A typical way of solving this problem is to upload your real file, and then to upload an empty 'done.txt' file.
The automated system should wait for the appearance of the 'done' file before trying to read the real file.
A simple file locking mechanism for SFTP is to first upload a file to a directory (folder) where the read process isn't looking. You can "make" an alternate folder using the sftp> mkdir command. Upload the file to the alternate directory, instead of the ultimate destination directory. Once the SFTP> put command completes, then do a move like this:
SFTP> move alternate_path/filename destination_path/filename. Since the SFTP "move" is just switching the file pointers, it is atomic, so it is an effective lock.
My application is keeping watch on a set of folders where users can upload files. When a file upload is finished I have to apply a treatment, but I don't know how to detect that a file has not finish to upload.
Any way to detect if a file is not released yet by the FTP server?
There's no generic solution to this problem.
Some FTP servers lock the file being uploaded, preventing you from accessing it, while the file is still being uploaded. For example IIS FTP server does that. Most other FTP servers do not. See my answer at Prevent file from being accessed as it's being uploaded.
There are some common workarounds to the problem (originally posted in SFTP file lock mechanism, but relevant for the FTP too):
You can have the client upload a "done" file once the upload finishes. Make your automated system wait for the "done" file to appear.
You can have a dedicated "upload" folder and have the client (atomically) move the uploaded file to a "done" folder. Make your automated system look to the "done" folder only.
Have a file naming convention for files being uploaded (".filepart") and have the client (atomically) rename the file after upload to its final name. Make your automated system ignore the ".filepart" files.
See (my) article Locking files while uploading / Upload to temporary file name for an example of implementing this approach.
Also, some FTP servers have this functionality built-in. For example ProFTPD with its HiddenStores directive.
A gross hack is to periodically check for file attributes (size and time) and consider the upload finished, if the attributes have not changed for some time interval.
You can also make use of the fact that some file formats have clear end-of-the-file marker (like XML or ZIP). So you know, that the file is incomplete.
Some FTP servers allow you to configure a hook to be called, when an upload is finished. You can make use of that. For example ProFTPD has a mod_exec module (see the ExecOnCommand directive).
I use ftputil to implement this work-around:
connect to ftp server
list all files of the directory
call stat() on each file
wait N seconds
For each file: call stat() again. If result is different, then skip this file, since it was modified during the last seconds.
If stat() result is not different, then download the file.
This whole ftp-fetching is old and obsolete technology. I hope that the customer will use a modern http API the next time :-)
If you are reading files of particular extensions, then use WINSCP for File Transfer. It will create a temporary file with extension .filepart and it will turn to the actual file extension once it fully transfer the file.
I hope, it will help someone.
This is a classic problem with FTP transfers. The only mostly reliable method I've found is to send a file, then send a second short "marker" file just to tell the recipient the transfer of the first is complete. You can use a file naming convention and just check for existence of the second file.
You might get fancy and make the content of the second file a checksum of the first file. Then you could verify the first file. (You don't have the problem with the second file because you just wait until file size = checksum size).
And of course this only works if you can get the sender to send a second file.
My basic requirement is to add a ".temp" suffix to a file while it is getting uploaded on to the FTP server (suffix should be only for time until the file is fully uploaded).
As per my understanding, i thought this could be achieved by: Add a suffix to file on the local machine, then upload it and after the upload is complete, rename the file to remove suffix on server.
But now the another problem is that i could not found a way to rename a file on FTP server using cocoa. I know renaming a file is feasible using Java or other languages but i want to achieve the same in objectiveC.
Please tell be if the above task is feasible and if it is what is the approach i should follow?
FYI: I know how to upload and download a file on FTP server using NSInputStream and NSOutputStream.
Thanks. I'd appreciate any help.
Take a look at CFNetwork FTP docs. They do not mention exactly your case, but there may be a way to sent RNFR and RNTO commands over the stream.
https://developer.apple.com/library/ios/#documentation/Networking/Conceptual/CFNetwork/CFFTPTasks/CFFTPTasks.html#//apple_ref/doc/uid/TP30001132-CH9-SW1
There is also a sample project you can look into:
https://developer.apple.com/library/ios/samplecode/SimpleFTPSample/Listings/Read_Me_About_SimpleFTPSample_txt.html
This library claims to do what you want (rename remote files)
http://www.chilkatsoft.com/ftp-objc.asp
Could anybody explain to me the process of uploading to and downloading form and ftp server with the iPhone SDK. If you could just point me in the right direction (e.g. documentation etc.). How difficult is a task like this?
Thanks in advance.
You can use this. It support all the basic ftp operations:
Download file
Upload file
Delete file
Delete directory
Create directory
List directory contents
[DISCLAIMER] I am the developer of the library, I needed a ftp library too in the past and came over this answer. However, I decided to write one myself because s7ftprequest didn't support at that point several operations that I needed.(like download or list directory)
Try this Simple FTP Download
The Apple documentation will provide far more info in general than I could. Have a look at:
http://developer.apple.com/iphone/library/documentation/Networking/Conceptual/CFNetwork/CFFTPTasks/CFFTPTasks.html#//apple_ref/doc/uid/TP30001132-CH9-SW1
which details the FTP information you need. If you prefer a PDF with all the networking info in it, have a look at:
http://developer.apple.com/iphone/library/documentation/Networking/Conceptual/CFNetwork/CFNetwork.pdf
In this, you'll be particularly interested in Chapter 5. Both detail working with FTP sites, including uploading, downloading, retrieving directory listings, etc.
s7ftprequest only for uploading files to FTP.
The below is sample code from apple
http://developer.apple.com/library/ios/#samplecode/SimpleFTPSample/Introduction/Intro.html
Limitations:
FTPS (that is, FTP over TLS)
deleting items
renaming items
other less common FTP commands
custom FTP commands
I ended up using GoldRacoon. It turns out that in iOS/objc land, there's BlackRaccoon as the original FTP client lib, then WhiteRaccoon was forked from that, and then GoldRacoon was forked from WhiteRaccoon (I think)
pod search GoldRaccoon
... will show you.
I ended up making a few changes (in my own fork) so that you can pass in a successBlock & failBlock into any request, so that block-y callers (like my classes) don't have extra work to manage the delegate callbacks. Github link (my fork): https://github.com/xaphod/GoldRaccoon
I have successfully been able to upload single files to an FTP with this:
My.Computer.Network.UploadFile
Is there a way to upload an entire directory, such as this:
My.Computer.Network.UploadFolder
Never worked with FTP in VB.NET, but it seems there is no direct way to do it.
Here is what people say about how it can be done:
Traversal the local directory
Create the same directory structure in the FTP server.
Upload the file in each local directory.
Also see here for same question answered (C#). It confirms there is no built-in way. You would have to write some code or use 3rd party libraries.