I'm running a kettle job which calls another job and then a transformation, the job runs smoothly in the PDI Spoon but when i call the job via a kitchen job it throws
2016/03/07 20:51:50 - Purifier_success - Executing command :
/home/ubuntu/dataintegration/data-integration/null/kettle_6a3db0e2-e4a6-11e5-b201-d7ec3590f1c5shell
2016/03/07 20:51:50 - Purifier_success - ERROR (version 5.4.0.1-130,
build 1 from 2015-06-14_12-34-55 by buildguy) : (stderr)
/home/ubuntu/dataintegration/data-integration/null/kettle_6a3db0e2-e4a6-11e5-b201-d7ec3590f1c5shell:
1:
/home/ubuntu/dataintegration/data-integration/null/kettle_6a3db0e2-e4a6-11e5-b201-d7ec3590f1c5shell:
2016/03/07 20:51:50 - Purifier_success - ERROR (version 5.4.0.1-130,
build 1 from 2015-06-14_12-34-55 by buildguy) : (stderr) : not found
But the job runs successfully. But its awkward to see to see the shell screen saying error.
Is there any way to solve it.
Related
In pentaho I get an error when I read a BigQuery table with a "Table Entry", I have these considerations:
This table was created from a Google Drive sheet with the service account
I can read this table with "Google Sheet Plugins"
[1]: https://i.stack.imgur.com/q9dl0.png
[2]: https://i.stack.imgur.com/gCxQK.png
2022/07/29 21:20:36 - Select.0 - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : An error occurred, processing will be stopped:
2022/07/29 21:20:36 - Select.0 - An error occurred executing SQL:
2022/07/29 21:20:36 - Select.0 - select 1 from `ms-data-warehouse.ms_Dev_Staging.ET_ods_hour`
2022/07/29 21:20:36 - Select.0 - [Simba][BigQueryJDBCDriver](100032) Error executing query job. Message: BIGQUERY_API_ERR
2022/07/29 21:20:36 - Select.0 - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : Error initializing step [Select]
2022/07/29 21:20:36 - insert drive - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : Step [Select.0] failed to initialize!
2022/07/29 21:20:36 - Select.0 - Finished reading query, closing connection.
2022/07/29 21:20:36 - Spoon - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : insert drive: preparing transformation execution failed
2022/07/29 21:20:36 - Spoon - ERROR (version 8.1.0.0-365, build 8.1.0.0-365 from 2018-04-30 09.42.24 by buildguy) : org.pentaho.di.core.exception.KettleException:
2022/07/29 21:20:36 - Spoon - We failed to initialize at least one step. Execution can not begin!
Your second screenshot says that it doesn't have the Drive access.
BigQuery doesn't keep the credential for accessing the Google Drive, instead, BigQuery uses the "current user" credential trying to access Google Drive.
Apparently the "service account" has the Google Drive access (in order to create that table) but either your account or the account used to setup the Simba BigQueryJDBCDriver doesn't have the access to the Google Drive file.
Here is my code in .travis.yml file，after run build is passing, but I found the TEST are failed. I have no idea about this and block me a few days, appreciate your help. THX
--- Error as below ---
2018-12-20 05:57:42.698 xcodebuild[2394:8186] [MT]
IDETestOperationsObserverDebug: (CB8F382A-5B77-4BDA-BC4F-6EDF7B7DB822)
Beginning test session aaaaTests-CB8F382A-5B77-4BDA-BC4F-6EDF7B7DB822 at 2018-
12-20 05:57:42.698 with Xcode 10A255 on target <DVTiPhoneSimulator:
0x7f86387e49d0> {
SimDevice: iPhone 6 (B25EC6DB-0B2F-4920-B6C2-8560331BA779, iOS 9.1, Booted)
} (9.1 (13B143))
2018-12-20 05:57:53.744 xcodebuild[2394:8186] [MT]
IDETestOperationsObserverDebug: 11.068 elapsed -- Testing started completed.
2018-12-20 05:57:53.744 xcodebuild[2394:8186] [MT]
IDETestOperationsObserverDebug: 0.000 sec, +0.000 sec -- start
2018-12-20 05:57:53.744 xcodebuild[2394:8186] [MT]
IDETestOperationsObserverDebug: 11.068 sec, +11.068 sec -- end
2018-12-20 05:57:53.746 xcodebuild[2394:8186] Error
Domain=IDETestOperationsObserverErrorDomain Code=6 "Early unexpected exit,
operation never finished bootstrapping - no restart will be attempted"
UserInfo={NSLocalizedDescription=Early unexpected exit, operation never
finished bootstrapping - no restart will be attempted,
NSUnderlyingError=0x7f86389df930 {Error
Domain=IDETestOperationsObserverErrorDomain Code=5 "Test runner exited before
starting test execution." UserInfo={NSLocalizedDescription=Test runner exited
before starting test execution., NSLocalizedRecoverySuggestion=If you believe
this error represents a bug, please attach the result bundle at
/Users/travis/Library/Developer/Xcode/DerivedData/aaaa-
bdscsxnorhzvygafnsqdiqoriugx/Logs/Test/Run-aaaa-2018.12.20_05-57-22-
+0000.xcresult}}}
Testing failed:
aaaa.app (2488) encountered an error (Early unexpected exit, operation never finished bootstrapping - no restart will be attempted. Underlying error: Test runner exited before starting test execution.)
** TEST FAILED **
The command "set -o pipefail && xcodebuild -workspace aaaa.xcworkspace -scheme aaaa -destination platform\=iOS\ Simulator,OS\=9.1,name\=iPhone\ 6 build test | xcpretty" exited with 65.
Done. Your build exited with 1.
---My code as below---
language: objective-c
osx_image: xcode10
xcode_workspace: aaaa.xcworkspace
xcode_scheme: aaaa
xcode_destination: platform=iOS Simulator,OS=9.1,name=iPhone 6
before_install:
- pod repo update
- npm install ios-sim -g
- ios-sim start --devicetypeid "com.apple.CoreSimulator.SimDeviceType.iPhone-6, 9.1"
The issue can fix by below step:
change the version of osx_image in .travis.yml file and retry
// if xcode10.1 build failed, you can change to other version, like xcode9.4
osx_image: xcode10.1
Travis support xcode version list:
https://docs.travis-ci.com/user/languages/objective-c/
I currently have a transformation that includes a UserDefinedJavaClass step. This runs as expected in Spoon. When I try and run this transformation as part of a web application using the Kettle jars I get the following error:
017/01/20 10:04:56 - Load questionnaires.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Error initializing UserDefinedJavaClass:
2017/01/20 10:04:56 - Load questionnaires.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : org.pentaho.di.core.exception.KettleException:
2017/01/20 10:04:56 - Load questionnaires.0 - null
2017/01/20 10:04:56 - Load questionnaires.0 - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Error initializing step [Load questionnaires]
2017/01/20 10:04:57 - evalue-di-risk - ERROR (version 6.1.0.1-196, build 1 from 2016-04-07 12.08.49 by buildguy) : Step [Load questionnaires.0] failed to initialize!
Because this works in Spoon, I can only assume I am missing a library. I am already including kettle-core, kettle-engine (kettle version 6.1.0.1-196) and janino which I would have thought was all I required to get this running.
EDIT I have taken every jar from the Spoon lib folder and dumped it into my webapp and this didn't work either.
EDIT AGAIN It turns out that the problem is having a UserDefinedJavaClass and a Table Input that contains variables which should be replaced during the transformation does not work and causes the above error.
you can not do that, you have to use the sdk api
and compile from your application.
http://wiki.pentaho.com/display/EAI/The+PDI+SDK
I am trying to run a simple Pig script and have scheduled it via Oozie , however , I get the following Oozie error after the script is run.
I am using Cloudera Enterprise Data Hub Edition Trial 5.6.0 (#54 built by jenkins on 20160211-1910 git: 1c2be84380aa23bd5d6993ec300e144c78b37bf2) .
> 2016-04-09 06:37:06,229 [uber-SubtaskRunner] INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.JobControlCompiler
> - Using reducer estimator: org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.InputSizeReducerEstimator
> 2016-04-09 06:37:06,237 [uber-SubtaskRunner] ERROR org.apache.pig.tools.grunt.Grunt - ERROR 2017: Internal error creating
> job configuration.
> <<< Invocation of Main class completed <<<
> Failing Oozie Launcher, Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
> Oozie Launcher failed, finishing Hadoop job gracefully
> Oozie Launcher, uploading action data to HDFS sequence file: hdfs://node.xxxx.com:8020/user/admin/oozie-oozi/0000000-160409060732867-oozie-oozi-W/pig--pig/action-data.seq
EDIT.
Additional log info by using the oozie command shell as follows.
oozie job -log 0000001-160409063446097-oozie-oozi-W -oozie http://xxxnode:11000/oozie
Gives only the following
63446097-oozie-oozi-W] ACTION[0000001-160409063446097-oozie-oozi-W#FirstJob] Launcher ERROR, reason: Main class [org.apache.oozie.action.hadoop.PigMain], exit code [2]
I am trying to access an Excel sheet and perform actions on it. When I am running it, I get the below error. It is not reading the input file.
2015/04/29 13:11:42 - INPUT.0 - Opening openFile #0 : /var/opt/UTM/V1.2/data/DAILY_inputs/working/Input_29-04-2015.xlsx
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : Unexpected error :
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : java.lang.NoClassDefFoundError: jxl/WorkbookSettings
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : at org.pentaho.di.trans.steps.excelinput.ExcelInput.getRowFromWorkbooks(ExcelInput.java:501)
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : at org.pentaho.di.trans.steps.excelinput.ExcelInput.processRow(ExcelInput.java:405)
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : at org.pentaho.di.trans.step.BaseStep.runStepThread(BaseStep.java:2664)
2015/04/29 13:11:42 - INPUT.0 - ERROR (version 3.1.0, build 826 from 2008/09/30 11:30:46) : at org.pentaho.di.trans.steps.excelinput.ExcelInput.run(ExcelInput.java*
As far as I know from your errors, there is a jar(Java library) file missing.
Could you do a quick check in Pentaho folder and look for this file.
.../data-integration/lib/jxl-2.6.12.jar
If it's missing, you could try to download and put it into 'lib' folder.
Download from here:
http://mirrors.ibiblio.org/pub/mirrors/maven/net.sourceforge.jexcelapi/jars/jxl-2.6.jar
Then restart Pentaho and try again.