WE have a call recordings everyday that are posted in FTP. As a backup i wanted to copy those recordings from FTP in to local drive on daily basis.Is it possible to copy entire folder as a whole and paste the folder in my local and rename it ?
thanks in advance
BIDS in 2008 has an FTP Task, but I never use it. There are also several third party extensions for SFTP and the like, but I don't use those.
[Edit: Since the above links are to CodePlex, which is shutting down, I'll provide more detail. The first link is to a project titled "SSIS SFTP Task Control Flow Component" and the second is to a project titled "SSIS Extensions - SFTP Task, PGP Task, Zip Task". As of June 2017 the projects do not appear to have an obvious rehosting.]
I tend to use WinSCP with a script file. It's certainly the least Microsoft-y approach, but, IMX, it's more foolproof. This has the advantage of working for FTP, FTPS, and SFTP. It has the disadvantage that you're storing the password in plain text. You can store it in the registry with WinSCP's session manager, but I believe that's a per-user registry which makes for lots of possible fun when you have the SQL Agent using an SSIS Proxy account.
Create your script file with the commands you want to use. Be sure to specify option batch abort and option confirm off. You also might want to specify option failonnomatch, but I haven't tested that one yet myself since I'm still on a slightly older version that doesn't support that option.
Mine tend to look like this:
# Set batch settings
option batch abort
option confirm off
# Connect
open sftp://user:password#sftp.server.com -hostkey="ssh-rsa 2048 00:11:22:33:44:55:66:77:88:99:aa:bb:cc:dd:ee:ff"
# Change remote and local directories
lcd "M:\SSIS\eSchoolPlus to Clever\Output"
# Transfer files
put -nopreservetime -nopermissions -transfer=ascii file1.csv
put -nopreservetime -nopermissions -transfer=ascii file2.csv
put -nopreservetime -nopermissions -transfer=ascii file3.csv
put -nopreservetime -nopermissions -transfer=ascii file4.csv
put -nopreservetime -nopermissions -transfer=ascii file5.csv
close
exit
The options on put are due to how the remote server works for the script I happened to grab. get works about the same.
Then you use an Execute Process Task. The executable is C:\Program Files (x86)\WinSCP\WinSCP.exe and the Arguments I use are:
-console -script="X:\Path\To\Script\WinSCPScript.txt" -xmllog="X:\Path\To\Script\WinSCP-!S-!Y-!M-!D-!T.log" -xmlgroups`
That creates a daily log of just the WinSCP information. The XML logs are more readable and useful than the older log format, IMO, which contains a lot of mostly debugging information.
You can also go the route of using the WinSCP .NET library with an Execute Script Task, but that's obviously a lot more effort. They do have a HOWTO for that, however.
Yes - you would use a File System Task and set the Operation to Copy Directory (before Microsoft changed it with Win 95, folders were called directories). You can change the name in the destination connection where you choose the destination folder. You can use an expression to add a date if you want.
https://msdn.microsoft.com/en-us/ms140185.aspx
Here's some good step-by-step instructions if you need them since Microsoft gives vague descriptions with no examples:
http://oops-solution.blogspot.com/2011/11/file-operation-copy-move-rename-and.htmlhttp://oops-solution.blogspot.com/2011/11/file-operation-copy-move-rename-and.html
We have multiple WinSCP processes to upload/download files from external servers. These jobs run to a schedule but can often overlap as they are running so frequently.
There are occasions where we can successfully upload a file to a server, however WinSCP exits as if it has failed, because it cannot write back to the ini file.
Error writting to file 'c:\progra~1\winSCP\WinSCP.ini'
System Error. Code: 32.
The process cannot access the file because it is being used by another process
It appears that this is due to two or more processes trying to write back to the ini file at the same time.
This is then causing us to treat the files uploaded as failures and re-upload them on the next run (not great when you're dealing with transactional data)
According to the Configuration Guide, we can set the properties of the WinSCP ini file to read-only:
Particularly when using shared INI file, you can set read-only
attribute to the INI file to prevent WinSCP from overwriting the file.
Before making this change, I was hoping someone could tell me the following:
What exactly gets written back to the file?
What issues could arise from setting the file to Read-Only?
Typically, no important data are written after script run, maybe some caches, statistics, etc. You can compare the INI file before and after the run to see yourself.
You can probably turn off all these to avoid WinSCP from writing them, but setting the INI file read only is more reliable and I would recommend it anyway. You would have no problems with that.
Though the best practice is not to rely on external configuration.
For example I have a client-server application, this application often gets updated (it's an exe file). If I download the update on the server machine then the same update should be transfered to the client machines, or vice-versa.
At the moment the update is downloaded on all machines individually. My idea is downloading the update should be done only on the server and I'm planning to make an option in the client to copy the *.exe file (update) directly from the server and paste it on the installation path.
How can I make this happen?
NOTE : the update is a self extracting file.
There is already a technology for achieving this called ClickOnce. The client application can be "published" to a share that is accessible to all the clients, then each time the client is executed a version check is done - if a later version is detected on the share then it is downloaded before execution continues.
You can read more about this here: ClickOnce Security and Deployment. Creating a ClickOnce package and publishing it is a feature already built into Visual Studio, so you do not need to write any code.
You have to write an application that will be split into different parts :
Detect file change either on the client or server machine
Perform the copy of the exe file to server or other clients.
In all cases you can't tell an exe file to update it self.
I am trying to write a Windows service on a PC running 64-bit Windows using Visual Studio 2008. In this service, I am trying to read a a control file from an external drive located on a different machine on the same LAN. The path to the file from the reading machine will be via a mapped network drive...( T:). I am using a TextFieldParser from Microsoft.VisualBasic.FileIO class to read the file at T:\filename. I'm getting a file not found exception, however, the path to the drive works perfectly if I copy and paste it into Windows Explorer from the same machine.
Anyone know if there are any issues connecting in this manner and/or what I am doing wrong?
Thanks for you help.
You will need to make sure that the account under which the service runs has a drive mapping to T:, or, better, try using a UNC path (e.g. \\server1\someshare\filename). And you will still need to make sure that the account has access to the file. Try to use an account with its access rights limited to only what it needs, so not the Administrator account.
Where would you write an error log file, say ErrorLog.txt, in Windows? Keep in mind the path would need to be open to basic users for file write permissions.
I know the eventlog is a possible location for writing errors, but does it work for "user" level permissions?
EDIT: I am targeting Windows 2003, but I was posing the question in such a way as to have a "General Guideline" for where to write error logs.
As for the EventLog, I have had issues before in an ASP.NET application where I wanted to log to the Windows event log, but I had security issues causing me heartache. (I do not recall the issues I had, but remember having them.)
Have you considered logging the event viewer instead? If you want to write your own log, I suggest the users local app setting directory. Make a product directory under there. It's different on different version of Windows.
On Vista, you cannot put files like this under c:\program files. You will run into a lot of problems with it.
In .NET, you can find out this folder with this:
Environment.GetFolderPath(Environment.SpecialFolder.LocalApplicationData)
And the Event Log is fairly simple to use too:
http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog.aspx
Text files are great for a server application (you did say Windows 2003). You should have a separate log file for each server application, the location is really a matter of convention to agree with administrators. E.g. for ASP.NET apps I've often seen them placed on a separate disk from the application under a folder structure that mimics the virtual directory structure.
For client apps, one disadvantage of text files is that a user may start multiple copies of your application (unless you've taken specific steps to prevent this). So you have the problem of contention if multiple instances attempt to write to the same log file. For this reason I would always prefer the Windows Event Log for client apps. One caveat is that you need to be an administrator to create an event log - this can be done e.g. by the setup package.
If you do use a file, I'd suggest using the folder Environment.SpecialFolder.LocalApplicationData rather than SpecialFolder.ApplicationData as suggested by others. LocalApplicationData is on the local disk: you don't want network problems to stop you from logging when the user has a roaming profile. For a WinForms application, use Application.LocalUserAppDataPath.
In either case, I would use a configuration file to decide where to log, so that you can easily change it. E.g. if you use Log4Net or a similar framework, you can easily configure whether to log to a text file, event log, both or elsewhere (e.g. a database) without changing your app.
The standard location(s) are:
C:\Documents and Settings\All Users\Application Data\MyApp
or
C:\Documents and Settings\%Username%\Application Data\MyApp
(aka %UserProfile%\Application Data\MyApp) which would match your user level permission requirement. It also separates logs created by different users.
Using .NET runtime, these can be built as:
AppDir=
System.Environment.GetFolderPath(Environment.SpecialFolder.CommonApplicationData)
or
AppDir=
System.Environment.GetFolderPath(Environment.SpecialFolder.ApplicationData)
followed by:
MyAppDir = IO.Path.Combine(AppDir,'MyApp')
(Which, hopefully, maps Vista profiles too).
Personally, I would suggest using the Windows event log, it's great. If you can't, then write the file to the ApplicationData directory or the ProgramData (Application Data for all users on Windows XP) directory.
The Windows event log is definitely the way to go for logging of errors. You're not limited to the "Application" log as it's possible to create a new log target (e.g. "My Application"). That may need to be done as part of setup as I'm not sure if it requires administrative privileges or not. There's a Microsoft example in C# at http://support.microsoft.com/kb/307024.
Windows 2008 also has Event Log Forwarding which can be quite handy with server applications.
I agree with Lou on this, but I prefer to set this up in a configuration file like Joe said. You can use
file value="${APPDATA}/Test/log-file.txt"
("Test" could be whatever you want, or removed entirely) in the configuration file, which causes the log file to be written to "/Documents and Settings/LoginUser/Application
Data/Test" on Windows XP and to "/Users/LoginUser/AppData/Roaming/Test on Windows Vista.
I am just adding this as I just spent way too much time figuring how to make this work on Windows Vista...
This works as-is with Windows applications. To use logging in web applications, I found Phil Haack's blog entry on this to be a great resource:
http://haacked.com/archive/2005/03/07/ConfiguringLog4NetForWebApplications.aspx
%TEMP% is always a good location for logs I find.
Going against the grain here - it depends on what you need to do. Sometimes you need to manipulate the results, so log.txt is the way to go. It's simple, mutable, and easy to search.
Take an example from Joel. Fogbugz will send a log / dump of error messages via http to their server. You could do the same and not have to worry about the user's access rights on their drive.
I personally don't like to use the Windows Event Log where I am right now because we do not have access to the production servers, so that would mean that we would need to request access every time we wanted to look at the errors. It is not a speedy process unfortunately, so your troubleshooting is completely haulted by waiting for someone else. I also don't like that they kind of get lost within the ones from other applications. Sure you can sort, but it's just a bit of a nucance scrolling down. What you use will end up being a combination of personal preference coupled along with limitations of the enviroment you are working in. (log file, event log, or database)
Put it in the directory of the application. The users will need access to the folder to run and execute the application, and you can check write access on application startup.
The event log is a pain to use for troubleshooting, but you should still post significant errors there.
EDIT - You should look into the MS Application Blocks for logging if you are using .NET. They really make life easy.
Jeez Karma-killers. Next time I won't even offer a suggestion when the poster puts up an incomplete post.