Password Encryption for PDI Job - pentaho

I am creating a batch file to schedule a PDI spoon job.
F:\pdi-ce-9.1.0.0-324\data-integration\kitchen.bat /rep:<RepositoryName> /job:ParentJob /dir: /user:<UserName> /pass:<Password> /level:Basic
The problem here is we need to provide the password in plain text format which I really want to avoid. I tried encr -kettle <password> and then using
F:\pdi-ce-9.1.0.0-324\data-integration\kitchen.bat /rep:<RepositoryName> /job:ParentJob /dir: /user:<UserName> /pass:Encrypted <EncyptedPassword> /level:Basic
Ref: Can not find schedule perspective in pentaho kettle spoon
But it didn't work.
Can anyone please tell me the right way to get it done?
Thanks in advance.

Related

Setting up and saving connection profile in SQL Workbench/j from command line/batch

Using the guide https://www.sql-workbench.eu/manual/commandline.html#commandline-connect-noprofile
to set up as profile and save the profile for future use.
This is possible manually via "7.2. Managing profile groups"
https://www.sql-workbench.eu/manual/profiles.html#profile-workbench-settings
Is it possible to save a profile via the command line?
SQLWorkbench64.exe -username=user -password=pass -url=jdbc.. -driver=Redshift driverJar=filepath
There is no direct support for this.
But you can use the WbStoreProfile command to store the currently active connection as a profile.
You can do this through the -command parameter.
SQLWorkbench64.exe -command="WbStoreProfile -name=test" -username=user -password=pass -url=jdbc.. -driver=Redshift driverJar=filepath
That will connect to the database, then run the WbStoreProfile command and exit again.
The profile will be saved under the name "test" in wb-profiles.properties.
But it's probably easier to edit wb-profiles.properties manually.

Getting the code of a Redis script using the sha value

Is it possible to return the script that was loaded into Redis through SCRIPT LOAD using the SHA value?
No, currently (v6.x) there isn't. There's an open PR (https://github.com/redis/redis/pull/4646) if you want to add your feedback there.

Jenkins. How to change file content

I've created a new project in Jenkins.
But now I want to change some information in my .properties file
How can I do that? (I've already read about Environment Injector, but I don't think, that it's what I need)
So, and one more questions - will this changes commit in file? (I don't want to do that)
Thank you!
Try using a scripted pipeline and you can use groovy for this task. For instance:
Properties props = new Properties()
File propsFile = new File(".properties")
props.load(propsFile.newDataInputStream())
props.setProperty('key', 'value')
props.store(propsFile.newWriter(), null)
You open the properties file, change the value of a specific key element and write to the same file when done.
If you are making use of Jenkins and it is on a windows machine. You can make use of PowerShell to change the file content
Here in this example,
I am actually trying to modify the string occurrence 'Memory Usage' with Jenkins build number
(Get-Content C:\proj\Jenkins\workspace\QA.I9.Api\Sample.txt).replace('Memory Usage', $env:BUILD_NUMBER) | Set-Content C:\proj\Jenkins\workspace\QA.I9.Api\Sample.txt
Thanks,
xyzcoder.github.io

Accessing S3 directly from EMR map/reduce task

I am trying to figure out how to write directly from a EMR map task to the s3 bucket. I would like to run a python streaming job which would get some data from the internet and save it to s3 - without returning it back to reduce job. Can anyone help me with that?
Why don't you just set the output of your MR job to be a s3 directory and tell it that there is no reducer:
./elastic-mapreduce ..... --output s3n://bucket/outputfiles --reducer NONE
That should do what you want it to.
Then your script can do something like this (sorry, ruby):
STDIN.each do |url|
puts extract_data(url)
end

Problem during SQL Bulk Load

we've got a real confusing problem. We're trying to test an SQL Bulk Load using a little app we've written that passes in the datafile XML, the schema, and the SQL database connection string.
It's a very straight-forward app, here's the main part of the code:
SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class objBL = new SQLXMLBULKLOADLib.SQLXMLBulkLoad4Class();
objBL.ConnectionString = "provider=sqloledb;Data Source=SERVER\\SERVER; Database=Main;User Id=Username;Password=password;";
objBL.BulkLoad = true;
objBL.CheckConstraints = true;
objBL.ErrorLogFile = "error.xml";
objBL.KeepIdentity = false;
objBL.Execute("schema.xml", "data.xml");
As you can see, it's very simple but we're getting the following error from the library we're passing this stuff to: Interop.SQLXMLBULKLOADLib.dll.
The message reads:
Failure: Attempted to read or write protected memory. This is often an indication that other memory has been corrupted
We have no idea what's causing it or what it even means.
Before this we first had an error because SQLXML4.0 wasn't installed, so that was easy to fix. Then there was an error because it couldn't connect to the database (wrong connection string) - fixed. Now there's this and we are just baffled.
Thanks for any help. We're really scratching our heads!
I am not familiar with this particular utility (Interop.SQLXMLBULKLOADLib.dll), but have you checked that your XML validates to its schema .xsd file? Perhaps the dll could have issues with loading the xml data file into memory structures if it is invalid?
I try to understand your problem ,but i have more doubt in that,
If u have time try access the below link ,i think it will definitely useful for you
link text
I know I did something that raised this error message once, but (as often happens) the problem ended up having nothing to do with the error message. Not much help, alas.
Some troubleshooting ideas: try to determine the actual SQL command being generated and submitted by the application to SQL Server (SQL Profiler should help here), and run it as "close" to the database as possible--from within SSMS, using SQLCMD, direct BCP call, whatever is appropriate. Detailing all tests you make and the results you get may help.