I am tasked to automate the process of downloading files from Amazon S3. I am quite new this, after searching on the web I found this command line utility for windows s3.exe
I run the get command with the below syntax
s3 get xxx-xxx/yyyy_yyy/ D:\ /key:****** /secret:*****
xxx-xxx(bucketname)
yyyy_yyy(keyprefix)
D:(local path)
When I execute this the only message i see is s3.exe version 2.0 - check for updates at http://s3.codeplex.com and then it takes me back to the folder where I run the command. Pardon me if this too basic a question for this forum but if anyone have used this utility some insights will help me achieve this
Related
How to get Data Dump from magento cloud. I have tried magento-cloud db:dump. It has given me .sql file. I dont know where to run that file.
I tried Magento CLI. I got sql file using magento-cloud db:dump in my terminal.
P.S. I am using Mac
I am not sure why this file cannot be found.
I am using Dagster to run the DBT Docs Generate command, upload those files to S3 so that they can be deployed via CI/CD to an ngnix server.
The documentation will sometimes load after a fresh deploy, but quickly will throw the above error. I can confirm, though, that the debug and docs generate CLI commands produce no error(s), and all the files (including the catalog.json) are present on the ngninx server.
I am not sure how to debug this weird issue. Thanks!
I tried running the docs generate command with the --no-compile flag, but it produces the same result.
I'm very new at this and need some help; I'm sure I'm not doing something right. I have a Synology NAS that has a cool options to sync files to Google cloud storage. This is a great way to get my backups off site
I have my backups syncing to a cold line storage bucket. Now that my files are syncing I'm looking to document the process if I need to retrieve them.
I want to download a whole folder and all of the files inside it to a windows server. I installed the gsutil and trying to run this command.
gsutil -m cp -R dir gs://bhp_backup_sync/backup/foldername
but after I run this I get the following exception.
CommandException: No URLs matched: dir
CommandException: 1 file/object could not be transferred.
NOOB here what am I missing?
I am new to WinSCP and am attempting to create a script file that will eventually be used with SSIS to download files from an SFTP site. A lot of the literature WinSCP includes explains the file downloading or uploading portions. For the time being, I just want to create a script to test the connection first and will build from there.
So far I saved the connection in WinSCP and have the following. The below code does not seem to function at all and I am not sure where else to go as I am still reading about the scripting for WinSCP. Is there a way or can someone point me in a direction to see if I am in fact connecting via through the script?
option batch on
option confirm off
open username#address
exit
Not sure what SSIS is (sorry) but I can tell you how I'd set it up from a windows batch file if that helps:
If you are open to using a different software, consider using cygwin. It mimics a linux shell so linux users on windows have a lot of linux utilities handy. That being said, there are some commands which can run on windows straight from command prompt (and thus batchable). What you'd need to do:
1) install cygwin
2) Create a "passwordless" login (using ssh-rsa authentication). To do this start your cygwin terminal and use the commands "ssh-keygen" and "ssh-copy-id" (more on that later)
3) Now you can run "sftp" from the DOS command prompt (does not require cygwin terminal) and sftp to your account. No password required because of step 2).
A few follow up info:
What can run from dos command prompt and what must be run from cygwin terminal?
If you go to the "bin" directory of cygwin (for me it's in c:\cygwin\bin) you can see all the cygwin utilities. Anything with "exe" extension can be run from dos command prompt. If no "exe" extension, must start cygwin terminal first
How to set up ssh-rsa authentication?
You can pretty much google "ssh login without password" and pull up a lot of results. This is common for setting up login from one linux system to another. You would be using the same steps using cygwin on windows. My instructions are here:
http://geekswing.com/geek/unix/how-to-ssh-login-without-a-password-using-ssh-keygen-quick-tutorial/
Storing session settings in WinSCP GUI and trying to access them from WinSCP script running in SSIS is generally a bad idea. I believe there's no example or guide on WinSCP site that would suggest doing that.
WinSCP stores its configuration in registry in HKEY_CURRENT_USER hive. The SSIS typically runs under a dedicated system account, that have its own HKEY_CURRENT_USER hive, and won't see the GUI configuration.
For details see WinSCP FAQ about your problem:
https://winscp.net/eng/docs/faq_scheduler
The best you can do is isolate your your script from configuration by using the session URL with the open command, instead of the stored site name.
See also https://winscp.net/eng/docs/scripting#configuration
Your actual problem can be completely different though. But that's hard to guess as you have not shared any details, such as error message, log file, etc.
I am using the 7zip standalone .exe to unzip a file. I am using the Execute Process task for this. I have tested this over and over again on multiple machines and I know it works (at least in debug mode/visual studio). I have uploaded this package the server. I have created a job that calls said package from the Package Store. The package is not able to find the .exe no matter where I put it.
My first thought was to put the .exe on the C:\ drive, which failed. I have also failed in my attempts to place the .exe on a network location that the account the package is running under has full control over.
Basically, has anybody else had issues getting the Execute Process Task to find an executable when the package is uploaded to the server?
The error message is
Can't find 7za.exe in directory C:\7zip
I'll risk a downvote for being wrong, but I believe you have a permission issue.
You say it runs fine on other servers from BIDS, try it without BIDS. Call it from a command-line on a box that it works on.
dtexec.exe /file C:\HereComesTheUnzipper.dtsx
If that works, then repeat the step on the troublesome server. RDC into the box and try again
dtexec.exe /ser localhost /sq HereComesTheUnzipper
If that still works, then you are looking at an issue with the job. What account is the SQL Agent service running as? Is the SSIS job step running as a particular set of credentials? If so, is it a SQL Server login (which wouldn't map to anything on the physical box)? Regardless of what your answer is, the resolution will be to ensure the account has access to
7z.exe
whatever scratch area 7zip may use while unpacking files (I assume %temp%)
the output folder (C:\bin\7z.exe -e e:\data\MyThing.7z)