Error while executing pentaho transformation (talking to Sql server) through java program - pentaho

I have a Kettle (5.4.0.1) transformation which executes query in MS Sql server 2008 R2 in "Table Input" task.
The task executes successfully in the Spoon UI directly or through Job.
Same transformation I want to execute through a Java program for which I got this code -
KettleEnvironment.init();
TransMeta metaData = new TransMeta("first_transformation.ktr");
Trans trans = new Trans( metaData );
trans.execute( null );
trans.waitUntilFinished();
But on executing it in a java program I get this error -
2015/07/30 20:08:34 - TestTransformation - Dispatching started for
transformation [TestTransformation] 2015/07/30 20:08:34 - XML Output.0
- Opening output stream in encoding: UTF-8 2015/07/30 20:08:34 - Table input.0 - ERROR (version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55
by buildguy) : An error occurred, processing will be stopped:
2015/07/30 20:08:34 - Table input.0 - Error occurred while trying to
connect to the database 2015/07/30 20:08:34 - Table input.0 -
java.io.File parameter must be a directory.
[C:\Root\EclipseWorkSpace\TestProject\simple-jndi] 2015/07/30 20:08:34
- Table input.0 - Finished reading query, closing connection. 2015/07/30 20:08:34 - Table input.0 - ERROR (version 5.4.0.1-130,
build 1 from 2015-06-14_12-34-55 by buildguy) : Error initializing
step [Table input] 2015/07/30 20:08:34 - TestTransformation - ERROR
(version 5.4.0.1-130, build 1 from 2015-06-14_12-34-55 by buildguy) :
Step [Table input.0] failed to initialize!
org.pentaho.di.core.exception.KettleException: We failed to
initialize at least one step. Execution can not begin! at
org.pentaho.di.trans.Trans.prepareExecution(Trans.java:1149) at
org.pentaho.di.trans.Trans.execute(Trans.java:607) at
TestKettle.main(TestKettle.java:24)
What could be the issue here as the database connection happens successfully while executing through UI and same KTR file I am trying to execute in above code?

It seems that the Table Input Step in the Ktr is not able to read the data.
Firstly, if you are calling a ktr file from Java code, make sure that the database driver jar files is properly imported/build inside the Java project. I mean import the "sqlserver" database jar file in the java project.
You may try reading this blog. I have used Maven to handle dependency of jars (my db was : postgresql). And it works :)
Hope this helps :)

For anyone else facing this issue - The problem was the folder Simple-Jndi from Pentaho installation directory was required to be placed in the project folder which is not obvious from the error message above although it is mentioned in the error.
2015/07/30 20:08:34 - Table input.0 - java.io.File parameter must be a directory. [C:\Root\EclipseWorkSpace\TestProject\simple-jndi]
After this, I got two other errors which were more clear and was about files missing in the project folder - ESAPI.Properties and validation.properties. I downloaded them from link given here. Then the program ran successfully.

Related

Get Error While Job executed by SQL JOB AGENT

I am getting below error while execute SSIS Package with SQL Server Agent. I have use administrator account(proxy) to execute this job.
SSIS Package contain data flow task. I have used ODBC connection as data source and OLEDB connection for data destination.
Data Souce :- AS400 JDE system & Data Destination - SQL Server 2019
SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on ODBC Source returned error code 0x80004005. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing.There may be error messages posted before this with more information about the failure.End Error DTExec: The package execution returned DTSER_FAILURE (1)
`
Here is the SSIS Log.
Here the some more details.
• SQL Edition: 2019 Enterprise
• OS: Windows 2019 server edition
Issue description:
• I have SSIS Package which connect to JD Edwards database via ODBC Connection
• I have Scheduled SQL Job and one of the step is using above package to load data form JD Edwards
• The job step throw error during execution. Below is the detail description of the error
Executed as user: {DomainName}{UserName}. Microsoft (R) SQL Server Execute Package Utility Version 15.0.2000.5 for 32-bit Copyright (C) 2019 Microsoft. All rights reserved. Started: 12:39:40 PM Error: 2020-06-11 12:40:02.18 Code: 0xC0047038 Source: Data Flow Task SSIS.Pipeline Description: SSIS Error Code DTS_E_PRIMEOUTPUTFAILED. The PrimeOutput method on ODBC Source returned error code 0x80004005. The component returned a failure code when the pipeline engine called PrimeOutput(). The meaning of the failure code is defined by the component, but the error is fatal and the pipeline stopped executing. There may be error messages posted before this with more information about the failure. End Error DTExec: The package execution returned DTSER_FAILURE (1). Started: 12:39:40 PM Finished: 12:40:02 PM Elapsed: 21.969 seconds. The package execution failed. The step failed.
'
Can Anyone Help in this ?
That log you're looking at must be incomplete. To get all error messages:
USE SSISDB
SELECT * FROM catalog.executions WHERE package_name LIKE 'NameOfYourPackage'
Copy the execution_id of the relevant execution (look at dates and times)
SELECT * FROM catalog.event_messages WHERE operation_id=[execution_id from previous query] AND message_type>110

Question about job using powershell steps then SQL DTSX File

I have a simple SQL export to an XLS file. To keep the file clean, I have powershell steps:
Deletes the current file
Copies a template to the location
then runs a SQL step running a DTSX file that does a simple query and exports to the file.
when I run the DTSX file step it works fine, however after I run the two powershell steps it somehow permanently breaks the DTSX step, even if I run the job at that step only.
The only thing I can think of is I had to do set-location C: on the two powershell steps to have the code work, after i changed this the DTSX step constantly fails till I build it again, upload the new file, then it works until I run all three steps again. Any thoughts? I guess I could just build this in Visual Studio instead of using Powershell.
I tried to make the DTSX clear out the spreadsheet before it exports but that always failed, not sure if this is the way to go instead?
Step 1 (works every time):
Set-Location c:
Remove-Item -Path ""
Step 2 (works every time):
Set-Location c:
Copy-Item -Path "" -Destination ""
Step 3 (only works if I run it before ever trying the two above steps)
Type: SQL Server Integrations Service Package
Package Source: File system
I made it using a simple Export Data task.
Error after running the powershell steps, when running the DTSX step:
Executed as user:. Microsoft (R) SQL Server Execute Package Utility
Version 11.0.6260.1 for 32-bit Copyright (C) Microsoft Corporation.
All rights reserved. Started: 12:00:05 AM Error: 2019-01-24
00:00:05.92 Code: 0xC0202009 Source: Data Flow Task 1
Destination - Query [32] Description: SSIS Error Code
DTS_E_OLEDBERROR. An OLE DB error has occurred. Error code:
0x80040E37. End Error Error: 2019-01-24 00:00:05.92 Code:
0xC02020E8 Source: Data Flow Task 1 Destination - Query [32]
Description: Opening a rowset for "Query" failed. Check that the
object exists in the database. End Error Error: 2019-01-24
00:00:06.00 Code: 0xC004706B Source: Data Flow Task 1
SSIS.Pipeline Description: "Destination - Query" failed validation
and returned validation status "VS_ISBROKEN". End Error Error:
2019-01-24 00:00:06.00 Code: 0xC004700C Source: Data Flow Task
1 SSIS.Pipeline Description: One or more component failed
validation. End Error Error: 2019-01-24 00:00:06.00 Code:
0xC0024107 Source: Data Flow Task 1 Description: There were
errors during task validation. End Error DTExec: The package
execution returned DTSER_FAILURE (1). Started: 12:00:05 AM
Finished: 12:00:06 AM Elapsed: 0.812 seconds. The package execution
failed. The step failed.

(Pentaho PDI):Thread: Couldn't find starting point in this job

Pentaho running job through command line
I have a pentaho job which runs successfully in pentaho but if i try to run the same through command line i get the error -(But It runs Successfully in one PC throws, error in another PC)
Here is my problem of the moment:
2016/12/12 16:52:30 - Start of job execution
2016/12/12 16:52:30 - ERROR (version 6.0.0.0-353, build 1 from 2015-10-07
13.27.43 by buildguy) : A serious error occurred during job execution:
2016/12/12 16:52:30 - Couldn't find starting point in this job.
2016/12/12 16:52:30 - ERROR (version 6.0.0.0-353, build 1 from 2015-10-07
13.27.43 by buildguy) : org.pentaho.di.core.exception.KettleJobException:
2016/12/12 16:52:30 - Couldn't find starting point in this job.
2016/12/12 16:52:30 - at org.pentaho.di.job.Job.execute(Job.java:532)
2016/12/12 16:52:30 - at org.pentaho.di.job.Job.run(Job.java:435)
2016/12/12 16:52:30 - Kitchen - Finished!
2016/12/12 16:52:30 - Kitchen - ERROR (version 6.0.0.0-353, build 1 from
2015-10-07 13.27.43 by buildguy) : Finished with errors
2016/12/12 16:52:30 - Kitchen - Start=2016/12/12 16:52:15.536, Stop=2016/12/12 16:52:30.652
Just had the same issue (9.1), but it disappeared after I deleted the "Start" step and replaced it with a new one. Looks like the "Start" step must be named "Start", so NO renaming it, even if it is allowed...
Aren't you trying to run a KTR file instead of a KJB?
Take a look:
http://forums.pentaho.com/showthread.php?58160-Couldn-t-find-starting-point-in-this-job
you should try pan, pan is PDI command line tools.
PDI Command line tools
sh /home/patel/Downloads/pdi/pdi-ce-9.0.0.0-423/data-integration/pan.sh /file:/home/patel/Documents/replication.ktr
Are you sure if you are using ****Start**** step while writing a job.
P.S. Each Pentaho job needs a start step to execute.
Claus's answer is right,if you rename you "Start" model,you should useing normal name "Start",I use this method fixed my problem.

Unable to launch spoon from intellij

I am trying to launch PDI spoon from intellij(running in windows).
/lib/.jar and /libswt/win64/ have been added.
PDI version - pdi-ce-6.1.0.1-196.
But it is crashing mid way with the following error:
ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.ui.xul.XulException: Can not locate Xul document [ui/spoon.xul]
2016/07/28 11:33:07 - General - at org.pentaho.ui.xul.impl.AbstractXulLoader.findDocument(AbstractXulLoader.java:520)
2016/07/28 11:33:07 - General - at org.pentaho.ui.xul.impl.AbstractXulLoader.loadXul(AbstractXulLoader.java:238)
How can I resolve this issue?
It's because spoon ui needs config files.Download pentaho-kettle source then copy assembly/package-res/ui to your root dir.It will work.

Error while generating tstsuite from commandline

While generating test cases using command line option I am getting the following error.
EvoSuite
Going to generate test cases for class: testgeneration.Multiplication
[MASTER] 14:03:51.716 [main] ERROR TestGeneration - Not possible to access to clients. Clients' state: Terminated with exit status 1. Master registry port: 14981
[MASTER] 14:03:51.803 [main] ERROR SearchStatistics - No statistics has been saved because EvoSuite failed to generate any test case
[MASTER] 14:03:51.904 [main] ERROR TestGeneration - failed to write statistics data
That can happen if EvoSuite's client process does crash. Why is crashing is hard to tell without further information. Could it be a bug in EvoSuite, or some weird combination of events.