I am using PDI 8.3 with repo database in another server.
In my expectation, if I do not define any log connections in the job properties, the job will not send any logs to the repo database.
However, when I run a job with kitchen.sh, it defines new database connection "live_logging_info" that points to "localhost:5432". Because PDI repo database is in another server, the job fails.
May I know how to define the default DB log connection? Thank you.
Under PDI 8.3 there should be a folder called simple-jndi. Within that folder there should be a file called jdbc.properties. In that file near the bottom there are settings for live_logging_info. By default it points to localhost:5432 but you can set it to any location. Or it can be another type of database (MySQL,MSSQL, etc).
The settings that are available by default are:
live_logging_info/type=javax.sql.DataSource
live_logging_info/driver=org.postgresql.Driver
live_logging_info/url=jdbc:postgresql://localhost:5432/hibernate?searchpath=pentaho_dilogs
live_logging_info/user=hibuser
live_logging_info/password=password
Related
Setup: A pretty standard data export SSIS package (SQL Server 2016 compatible), created in VS2019/Data Tools and deployed using the SSIS Project Deployment model to the Integration Services Catalog of a SQL Server 2016 instance. The package creates files in a network folder before sending the file out via FTP and putting a copy of the file in a Sent folder.
The project requirements include having the package running on a schedule using "default" parameter values, as well as allowing users to manually run the package using "non-default" parameter values from within a stand-alone application.
Current behavior: the package behaves correctly when run from a SQL Server Agent Job that is configured with a SQL proxy and credentials mapped to a domain login with the proper permissions for the network folder.
Problem: the Data Flow task fails to create the file with a "Cannot open the datafile" error when running the package directly using any of the following methods (even when the "current" session is using the same credentials as the SQL Server Credentials/Proxy used by the SQL Server Agent Job):
Using SSMS to right-click on the package and selecting Execute
Using the DTEXEC SQL utility
Using the SSISDB.catalog.start_execution SQL Server stored procedure
As far as I'm aware, these are the only methods capable of starting a SSIS package and changing the package's parameter values. I either need to get one of the latter 2 methods to work, find another option that allows for changing the parameter values while launching the package, or use one of 2 techniques I'm aware of (detailed below) that would add yet another failure point to the process as well as other potential issues.
Note: If the process is changed to initially create the file on the SQL Server's local harddrive, then the Data Flow task succeeds, but the later copy to Sent folder task fails with a very similar permissions error.
Alternative #1: this technique requires creating a new table, loading the parameter values to the table, changing the package to check the table and potentially set it's parameters/variables based on what it finds. The package can then be launched using a SQL Server Agent Job (for which there are multiple methods to manually launch them) and if the calling object has correctly populated the table, the package will behave as if it's parameters were changed at runtime otherwise it will run with the default values.
Alternative #2: Change all folders used by the package to point to folders local to the SQL Server instance and then create a separate scheduled task/application/whatever, with the valid credentials, that would synchronize or move the files to their proper network folders.
even when the "current" session is using the same credentials as the SQL Server Credentials/Proxy used by the SQL Server Agent Job
This is probably because the account is not logged on locally at the SQL Server, and so it's a Double-Hop Impersonation scenario, and would require Kerberos Constrained Delegation to be configured.
And you are correct in assessing the options. The general solution is to invoke catalog.start_execution from a session running on the SQL Server, and an Agent Job is the simplest built-in way to do this (the others being xp_cmdshell, Service Broker Activation, or SQL CLR).
Hello I am trying to create a new plan on SQL server to backup all my database.
My goal is to backup them to a network drive thus if I do have some trouble with my server, I will be able to restore databases to other server thanks to backup present in the network drive.
When my plan is executed, I do have some error so I try to execute manually the relative query.
After some investigation, it seems even net use command doesn't work (whereas it is working and I do it from cmd)
EXEC XP_CMDSHELL 'net use Z: \\ServerName\loggin/user:loggin password'
error is
System error 1450 has occurred. Insufficient system resources exist to complete the requested service.
Beside, I do have another server where it is working so I suppose some configuration missing but can't find them
as my network drive is also accessible via FTP, I chose this way to make the job : create a batch file that run winscp and use this batch file in a SQL agent job . I need to add right to batch file to SQL Server agent account. I also need to define a credential and a proxy to be used in the job.
I am installing pentaho 8.1 CE on ubuntu 16.04
have made change of bd from HSQLDB to Mysql, tables have been created, jackrabbit, hibernate and hibernate.
When starting the server, not login, I imagine that the users are missing or not created. ??
the error that throws me in
catalina.out
ERROR [CompositeUserRoleListService] User / Role List could not be obtained.
java.lang.IllegalStateException: Target of Bean was never resolved: org.springframework.security.core.userdetails.UserDetailsService
atorg.pentaho.platform.engine.core.system.objfac.spring.BeanBuilder$1.invoke(BeanBuilder.java:159)
at com.sun.proxy.$Proxy84.loadUserByUsername(Unknown Source)
..
..
Database Jackrabbit no created tables..
any idea?
When changing the back-end database, there's a few things you need to make sure you check.
repository.xml
quartz.properties
hibernate-settings.xml
[your-database].hibernate.cfg.xml
context.xml (in WEB-INF)
You'll need to confirm the settings for the connection to the new database has been properly configured in all of these different config files. Some additional details on how to config these files for MySQL specifically can be found in the documentation here: https://help.pentaho.com/Documentation/8.1/Setup/Installation/Archive/MySQL_Repository
Past that, make sure that you delete the "repository" directory inside of /pentaho-solutions/system/jackrabbit as this is an index of the repository. If you change your database back-end, then this index needs to be rebuilt. The index is rebuilt automatically if the server sees that "repository" directory doesn't exist at startup.
I've found the same issue on Windows Server 2016 and MySQL8.0, with Pentaho 8.0.2.
Starting Pentaho server from the start-pentaho.bat command, everything was fine.
The problem arise when I was using Tomcat.
The solution was to be sure that Tomcat service was running with the LOCAL SYSTEM account.
If the Tomcat service runs with lower privileges, maybe it can't access to the pentaho-solutions directory and it cannot load the required java beans.
if everything as per the document is configured correctly, then You might be missing the connection change in applicationContext-spring-security-hibernate.properties, please check that.
I am trying to deploy my SSIS package and stepping into the error create master key in DB or open the master key in the session
I have changed the password sensitivity to be 'Dont save sensitive' for the project and package
Right Click on Projects(Integration Services Catalog ) and run through the wizard, this errors comes in the last step.
Ok I figured out how to solve this.
Drop the master key and create a new one if there are no other packages in your SSISDB. If there are other packages you should find a way out.
Save the master key in a back up location on the server and copy the password for later use.
The next is create service accounts with permissions to mitigate the keys across domain controllers.
Use the service account and change the logon for the SSIS,SQL Server Engine and Sql server Agent in the Windows services GUI.
Restart all the services mentioned above.
This will resolve the issue.
I have an SQL 2005 SSIS package that takes data from an Oracle DB Table, and transfers it to a SQL Server Table.
I have set up an "Oracle Provider for OLE DB" for the Oracle connection and a "SQL Native Client" for SQL Server Connection.
The Oracle and SQL connections will depend on the development and shipping stage, which are:
Local environment
SYS - For integration and System testing
UAT - For user acceptance testing
PRE - Mimics the LIVE system for confidence testing
Live - The live system
In the Connection Manager for Oracle, it expects the following:
Server Name (which, for example can be DEVSERVER)
User Name (which, for example, can be devserver_user)
Password (which, for example, can be devserver_pass)
So, I was wondering how I could parameterise these such that the settings are picked up depending on the server. Ideally this would be a connection string that is stored in the registry (to have commonality with the architecture of other systems in our company).
I have attempted to specify the above settings through Package Configurations. I have also tried specifying the connection string, which would look something like this:
Provider=OraOLEDB.Oracle;Data Source=DEVSERVER;User ID=devserver_user;Password=devserver_pass;PLSQLRSet=1;OLE DB Services = -2;
I have tried this through a registry setting, environment settings, and XML config file. I am mapping these item to the properties on the connection object, but the settings do not seem to hold. I.e. when I open the connection object these settings are not there.
What happens is that when I open the OLE DB source and specify the connection, it fails, because the connection object is not picking up the items in the Package Configurations.
Is there something I am missing, some setting that I have to configure. I guess I'm not sure as to what I'm not seeing anything!
Any help would be appreciated.
Just worked this out this myself.
This really was a case of RTFM! The first paragraph on the MSDN Package Configurations page says it all:
Typically, you create a package set
properties on the package objects
during package development, and then
add the configuration to the package.
Still, I hope this is still of help to other RTFMers!