Use of FTP "append" command - error-handling

I want to upload a file to a ftp server programmatically (C++). If the connection is lost while uploading a file, I wouldn't want to upload the file from scratch, but to upload only the part that I haven't sent.
Does the APPE command fulfill my demand? What list of FTP commands should I use exactly? And how?

I am googling details about APPE FTP command, what actually it does but most site just state only append. Then I try out the command to make sure it behave as expected.
I designing FTP auto sender that is used to send a log file from a machine to a server for reporting. I only want to send the last line of the log file.
When using a APPE command, it actually append the whole file content and append to the existing one in the server. This will cause the line entry duplicated.
The answer:
To do the resume of file if the last transfer is failed, there is no such command for that, but we need to use a sequence of command to achieve it.
The key point here is seek your local file to the last uploaded byte if you are using APPE command or using command REST. REST will start transfer on that particular byte start position. I end-up with this solution to perform after connection established:
Use APPE (I got the idea from FileZilla log):
Use SIZE to check for file exist and use it as resume marker.
Open local file system and seek on the marker.
Use APPE to upload and FTP server will append it automatically.
Use STOR with REST (I got the idea from edtFTPnet):
Use SIZE to check for file exist and use it as resume marker.
Send REST with the result you get from SIZE to tell FTP server to start write on the position.
Open local file system and seek on the marker.
Use STOR as normal upload.
Note that not all FTP server support for both way. I see FileZilla switch this two way depending on the server. My observation shows that using REST is the standard way. Download can also use REST to start download on the given byte position.
Remember that using resume support for ASCII transfer type will produce unexpected result since Unix and Windows have different line break byte count.
Try to manipulate FileZilla to see the behave in the log.
You can also check this useful open source FTP for .NET library on how they do it.
edtFTPnet

Check the RFC and specifically the APPEND command:
This
command causes the server-DTP to
accept the data transferred via the
data connection and to store the data
in a file at the server site. If the
file specified in the pathname exists
at the server site, then the data
shall be appended to that file;
otherwise the file specified in the
pathname shall be created at the
server site.
Note that you cannot simply APPEND the same file again. You should send the bytes remaining. That is, continue at the same position when the connection was lost.

Related

Remove portion of a remote file using SFTP

I would like to remove a portion of a file using the SFTP protocol. Example:
This is a sample text. --> This is a text.
The standard protocol write operation takes an offset and a string of data as inputs. However, the interface does not specify a file length to possibly remove characters from the file. That way the file can only ever grow in size. In the above example, if I attempted to update the file using write, the resulting output would be:
This is a text.e text.
What is the proper way of removing characters in SFTP? Is there perhaps a terminating character which is used to signal the end of a file when using write? Or do I have to just delete the entire file and re-upload a new one?
You cannot remove a part of a file contents. Not even from a local file, let only from a remote file over SFTP.
You have to overwrite the file – or at least whole part of the file starting with the first changed byte. And then truncate the file, in case the new contents is shorter than the original.
You can use copy-data SFTP extension to copy data within the file itself to avoid need to reupload the existing contents. If your SFTP server supports the extension. In OpenSSH, it's supported only in very recent version 9.0.

Doing DSPSMTF to display a stmf on browser but it all junk and it is downlading the file instead of displaying it. Also any idea about CONTTYPES file?

I am using CGI DSPSTMF command to display stmf file on web browser. I am copying a spool file to a stmf file using CPYSPLF *STMF option. Once copied i am passing IFS location to DSPSTMF command but it is going to download automatically and when i open the download file i am getting all Junk data any idea why?
Also, i noticed it is using CONTTYPES file in CGILIB and on my server it is empty. What should be the values in it and what should i do show correct data instead of junk. I tried to use different methods to copy the file to IFS like used cpytostmf instead of cpysplf but on IFS file looks correct not the download version.
What CCSID is the resulting stream file tagged with?
use WRKLNK and option 8=Display attributes
If 65535, that tells the system the data is binary and it won't try to translate the EBCDIC to ASCII.
The correct fix is to properly configure your IBM i so that the stream file is tagged with it's correct CCSID.
Do a WRKSYSVAL QCCSID ... if your system is still set to 65535, that's the start of your problem. But this isn't programming related, you can try posting to Server Fault but you might get better responses on the Midrange mailing list

FTP Server Download Returns blank files

We have a process in place built on Excel VBA that uploads a file to FTP Server. On the other side, our client downloads it. Very randomly, they complain that the file they received is blank (the file name is the same though). We then check at our end and see that the file that was uploaded was never blank. So here comes the problem: we're always arguing whether it was our error or theirs.
I figured that there might be a couple of reasons behind it but I have a few questions to ask before coming to conclusions:
If, say, the file was never uploaded (a possibility), what happens when the client runs a download process at their end? Can that download process generate a blank file with the same name as our output file? It sounds impossible to me but since the client is following up on this issue, I have to ask this silly question.
How does the mechanism work - what are the steps that happen on FTP server the moment my process completes uploading the file? I sometimes see that as soon as I upload the file, a 0kb file is created and then a second later (or less) the file with right size appears? Could it be possible that their process is running right before this actual file creation?
Thank you in advance for your help!

What's My.Computer.Network.UploadFile behavior on duplicate filename?

I have been given a program that uploades pdf files to an ftp server, which is something I never did. I've been asked what the behavior regarding attempting to upload a duplicate filename is. It apparently doesnt check for duplicate filenames manually, but the comand that uploads the file is My.Computer.Network.UploadFile and I can't find what happens when attempting to upload a duplicate file anywhere, does it throw an exception or overwrites the file?
It looks like My.Computer.Network.UploadFile is a wrapper around WebClient.UploadFile, and the documentation for that states:
This method uses the STOR command to upload an FTP resource.
In the FTP RFC 959 it says (I highlighted the relevant part):
STORE (STOR)
This command causes the server-DTP to accept the data
transferred via the data connection and to store the data as
a file at the server site. If the file specified in the
pathname exists at the server site, then its contents shall
be replaced by the data being transferred. A new file is
created at the server site if the file specified in the
pathname does not already exist.
So, if everything is following standards (and that part of RFC 959 hasn't been replaced, I didn't dig further!), then it should replace the existing file. However, it is possible for the server to deny overwriting of existing files, so the behavior is not guaranteed.
Of course, the best thing to do would be to try it out in your environment and see what it does.

Update Copy of Text File when Original Gets Modified

Trying to write a fairly simple program that reads line by line a file from an ftp server, stores each line in a string, then writes that string to a local file on my drive. Now this is fairly easy and worked perfectly.
This is the part that gets tricky. Every time the ORIGINAL file on the ftp gets updated (a new line is added), I want that line to be added into my LOCAL file created earlier. I do NOT want to recopy the whole file contents if the files are different. I simply want to add the missing lines.
Is there any way this can be done ? Any tips will be appreciated. You don't have to submit a code. Let me know of how this can be done logically and I will try to write the code for it.
Regards
You can't read a file "line by line" from an FTP server via FTP itself.
FTP is short for File Transfer Protocol, which is for transferring files. There's no way to do anything else via that protocol. If you want to access it "line by line", you're going to have to use something other than FTP.