We have a process in place built on Excel VBA that uploads a file to FTP Server. On the other side, our client downloads it. Very randomly, they complain that the file they received is blank (the file name is the same though). We then check at our end and see that the file that was uploaded was never blank. So here comes the problem: we're always arguing whether it was our error or theirs.
I figured that there might be a couple of reasons behind it but I have a few questions to ask before coming to conclusions:
If, say, the file was never uploaded (a possibility), what happens when the client runs a download process at their end? Can that download process generate a blank file with the same name as our output file? It sounds impossible to me but since the client is following up on this issue, I have to ask this silly question.
How does the mechanism work - what are the steps that happen on FTP server the moment my process completes uploading the file? I sometimes see that as soon as I upload the file, a 0kb file is created and then a second later (or less) the file with right size appears? Could it be possible that their process is running right before this actual file creation?
Thank you in advance for your help!
Related
I am using CPANEL and there I edited and created my website directly.
I had php file with about 300 lines and somehow I changed encoding to something other than utf-8. When I changed it and saved it saved only first 10-20 lines of my file. I cannot redo (ctrl+z) to bring my code back.
So my question is how to get lost file back? I haven't closed it yet so session is still valid. Please fast help. I know I will get downvote but I am in despair....
You may be lucky depending on backups...
If you go into the Files section and click Backups, then select the most recent backup where you know your file still existed in the correct state (before you made the encoding change).
If you then save the file and unzip it, hopefully your file will be there and you will be able to re-upload it. Good luck!
In future, definitely develop offline and use source control then upload copies of your file to avoid this.
So we have a client that creates "training packages" and then uploads them via ftp to their website. They create the training packages in PowerPoint, and then use some program to convert them into html/swf files and package them within a folder. When they upload, they use Filezilla, and just transfer the entire folder over. The folder is uniquely named, uses no spaces or special characters.
These files have uploaded fine for about a year. Recently, they've run into a problem. Whenever they try to upload training package folder, they are immediately presented with the "This file already exists, do you want to overwrite?" message. Except... the folder they're moving is brand new, and the file it's asking to overwrite DOESN'T EXIST. When they choose "Overwrite" the file looks like it transfers, but the file size is wrong, and the training package doesn't work correctly.
This happens with every training package they try to upload. It's not just a badly outputted package. Also, it's always the same file that has the problem--it's the main "player" for the training package, and though it contains different content for every package, it is the same file name (cplayer.swf) every time.
Things they've tried without success:
-Re-uploading the file again by itself, and overwriting
-Deleting the "bad" file and re-uploading the single file - Get the overwrite message again, even though the file DOES NOT EXIST.
-Renaming the file on the server and re-uploading the single file - Get the overwrite message.
-Renaming the single file locally within the package and uploading/renaming it - Won't let us rename because the file already exists.
-Used another FTP client - Same results as above, so not a client specific problem.
-Used a different FTP login - Same results as above, so not a permissions problem.
Other things of note:
-The file is small--it's not a time out problem. Plus, all other files upload fine, and some are a lot larger.
-They've emailed this file to me, and I've uploaded it successfully.
I am completely at my wits end. Does anyone have any ideas where I can at least troubleshoot a little further?
Thanks for the non-help, the downvote, and the general lack of response on what was a pretty serious issue for me.
In case anyone else has a similar problem, here's what was going on:
Virus software (specifically Malware Bytes) was blocking THIS ONE SINGLE FILE. All I had to do was exclude the folder that contained the file.
This is a weird persistant issue I am having with a small visual basic program.
Problem: The excutable of the vb program crashes when trying to run but when I step through the program inside of visual studio it runs successfully.
Details: The program performs a winscp.com transfer from a ftp server and then takes the downloaded file and extracts the data from it before sending it to a webpage. The program also decrypts a des3 encrypted file which holds the login details for the sftp server and the webpage.
My Thoughts: I was thinking this could be something to do with the excutable jumping ahead of the slower transfer and decrypt functions thus causing a "file not found" exception to be produced and the program to exit. Lending to this if I slowly (aka spend 10+ seconds stepping through the code) move through the code it works just fine.
Solution: I was thinking of including some kind of checks inside the program to make sure that the file exsists before moving on but I still wanted to get your guys opinion.
Thanks!
As requested
Crash Messages:
(I capture everything in exceptions and exit properly so no "crash" is reported but the error I get in my logs is as follows:)
1st run with standard test case:
Could not find file 'C:\Users\Administrator\Desktop..\ILC2INFOENC.txt'
2nd run standard case (the file above was not deleted by my cleanup function either as it could not be found)
Could not find file 'C:\Users\Administrator\Desktop..\ILCNETSL10663.csv'
Background:
The first file that is found missing is the encrypted login information file while the second file is the winscp.com downloaded csv file.
Thanks again for all the help and suggestions.
Solution: Applied checks to the existance of the files in question as to keep the vb program from jumping ahead of the slower moning openssl decrypt and the winscp file transfer. If I had the time to redo this portion of code I would have utilized the .net framework provided sftp function and the decrypter instead of winscp and openssl as to better control the flow of the program! If you need more options check below the original question for different takes of this issue.
Remote clients will upload images (and perhaps some instructional files in specially formatted text) to a "drop folder." Once the upload is complete we need to begin processing these images. It would be an easy, but flawed, solution to just have a script automatically begin processing any files in the folder every few seconds (the files can be move out of the folder once processed); but problems would arise when attempting to process large images which are only partially transfered.
What are some tricks I can use to ensure the files are fully uploaded before processing them?
A few of my own thoughts:
The script can check the validity of the file; ie, a partial jpeg would result in an error and you could respond to that error in the script, this would be fairly CPU intensive though. Some files have special markers on the end, but I can't count on this, I'm not sure what formats I'll be dealing with.
I've heard of "file handles" but haven't really figured out the basics of what they are and how I can tell if there is a "file handle" on a particular file. Basically the FTP daemon (actually, I'm on Windows, so "service") would keep a "handle" on the file while it's being uploaded and you would know not to process that file. These are just a few of my thoughts but I'm not really sure if they will work or if there are better or more accepted ways of solving this problem.
If you have an server-side script upload system (PHP, ASP, JSP, whatever), you could instruct the script to call another script to process the files, or to create a flag-file indicating the upload is done, something like this.
If your server is Linux-based, you can use lsof to check if the file is open. As your ftp/script/cgi will close the file after upload completes, lsof will not show the file in the list.
If your server is Windows-based, you can use Process Explorer to list the open files.
By what method are your users uploading the images?
Currently working on an application, that help you to take backup of the files in you machine at the server (hosted by the company itself), so that you can recover data after any hdd crash. I have implemented Single Instance feature, across the users.
Single Instance : A file uploaded already at the server, wouldn't be uploaded again. Whenever any other instance of the exact file uploaded will not be actual upload but some database changes and linked to the same previously uploaded file.
Issue arise when same file (that has not already been uploaded before) is uploaded simultaneously by more than one users, On Start file wouldn't be detected for an instance (as database is updated only after successful upload/backup). All are running, at once. What will be the best way to implement single instance in this way.
I am thinking when I let all the instance upload as it is. So more than one instance of the file will reside at the server. But whenever another backup of the same file will be taken afterwards, I will remove all the previous instances and link them up with the one. This will not let user double uploads and also less complex on the cost of some disc space that too for a while probably (till next upload of the same file will be done)
Thanks for your thoughts in advance.
Calculate the hash (signature) of the file before upload and store it in the DB.
Then - start uploading.
if a similar file will be mark for uploading during the upload of the first file (you will know b/c you already saved the hash) - you will hold the 2nd file upload, until the first one finish successfully, and then link, if the 1st on fails, you can go to the 2nd source and upload it.