I am currently trying to develop simple parameter passing process using Pentaho and execute the job from web (Carte). I have a transformation and also a job. I have successfully pass the parameter if I execute it directly through transformation. http://cluster:cluster#localhost:8080/kettle/executeTrans/?trans=/data-integration/PENTAHO_JOB/test_var.ktr&testvar=1234567
however, when I try to put the transformation in a Job and execute it in job level, I could not get the parameter testvar now even though I can run it successfully. i also found out that there is no Get Variable function in Job level. I wonder if I can get the parameter testvar by executing from job level in Carte?
http://cluster:cluster#localhost:8080/kettle/executeJob/?job=/data-integration/PENTAHO_JOB/test_var.kjb&testvar=1234567
#Raspi Surya :
Its working for me . You need to set the variable in parameter at job level. I used the below URL.
http://localhost:8080/kettle/executeJob/?job=C:\Data-Integration\carte-testing.kjb&file_name=C:\Data-Integration\file1.csv
See the attached SS
Related
I am calling a bigquery procedure using BigQueryInsertJob operator using "declare out_var String; call bq_sp("input params", out_var)". This is getting called properly and executing the stored procedure properly also. But what I need to do along with that, is to print the value of out_var to the composer log, to catch the value of out_var, so that I can do some more things depending on the value of that variable. Could you please help me, how can I catch the value of the OUT parameter value from the cloud composer. Thank you.
I have a stored procedure with input and output parameters. and calling that from composer.
This is working absolutely fine.
I am expecting to fetch the OUT parameter value in the cloud composer job
I have a requirement where I need to set assignee's to all the "user-tasks" in a process instance as soon as the instance is created, which is based on the candidate group set to the user-task.
i tries getting the user-tasks using this :
Collection<UserTask> userTasks = execution.getBpmnModelInstance().getModelElementsByType(UserTask.class);
which is correct in someway but i am not able to set the assignee's , Also, looks like this would apply to the process itself and not the process instance.
secondly , I tried getting it from the taskQuery which gives me only the next task and not all the user-tasks inside a process.
Please help !!
It does not work that way. A process flow can be simplified to "a token moves through the bpmn diagram" ... only the current position of the token is relevant. So naturally, the tasklist only gives you the current task. Not what could happen after ... which you cannot know, because if you had a gateway that continues differently based on the task outcome? So drop playing with the BPMN meta model. Focus on the runtime.
You have two choices to dynamically assign user tasks:
1.) in the modeler, instead of hard-assigning the task to "a-user", use an expression like ${taskAssignment.assignTask(task)} where "taskAssignment" is a bean that provides a String method that returns the user.
2.) add a taskListener on "create" to the task and set the assignee in the listener.
for option 2 you can use the camunda spring boot events (or the (outdated) camunda-bpm-reactor extension) to register one central component rather than adding a listener to every task.
I have one Execute SQL Task (DOE Params) with script:
select
SystemCode
,[Group]
,Process
,InputFolder
,InputFileName
,TargetFileName
from rpt.v_cfg_Report_DOE_Parameters
where
Report_Id = ?
and Procedure_Id = ?
And after there is new sql task (RUN DOE CALLER) which taking results from previous query, call procedure, which continue to process data for some another steps. I need to make File System Task which will copy that params from (GET DOE Params) to file and to save that file in folder where I can see which parameters are forwarded to procedure in a purpose of check.
Any idea how to do that?
You can use the result set from that query to an object variable. Then use a for each loop container set to ado.net iterator. This allows you to set the parameters to package variables for each time through. Those then drive the file system task. You could also use that query in a source component rather than a sql task. In the data flow you choose a recordset destination that points to the object variable.
I have a oozie workflow wchich consists of a sub-workflow. My main workflow takes three sqoop job names at a time in fork. Then it has to pass those names to the subworkflow. In main workflow there are three shell actions which receive values of job names in three respective variables(${job1},${job2},${job3}) . But my sub-workflow is common for all three shell actions. I want to assign the value of ${job1} to ${job}. Where to create the property ${job} and how to transfer the value of ${job1} to ${job}???? Please help.
Use a java action in between along with capture-output so that you can do whatever assignment or name change logic there.
Java will accept job1 and will output job=job1 using capture-output which in turn you may pass to sub-workflows.
I use the following command to load data.
/home/bigquery/bq load --max_bad_record=30000 -F '^' company.junelog entry.gz country:STRING,telco_name:STRING,datetime:STRING, ...
It has happened that when I got non-zero return code the data was still loaded. How do I make sure that the command is successful or not? Checking return code does not seem to help. There are times when I loaded the same file again because I got an error but the data was already available in bigquery.
You can use bq show -j of the load job and check job status.
If you are writing code to do the load, so you don't know the job id, you can pass the job id into the load operation (as long as it is unique) so you will know which job to check.
For instance you can run
/home/bigquery/bq load --job_id=some_unique_job_id --max_bad_record=30000 -F '^' company.junelog entry.gz country:STRING,telco_name:STRING,datetime:STRING, ...'
then
/home/bigquery/bq show --j some_unique_job_id
Note if you are creating new tables for every load (as opposed to appending), you could use the write disposition WRITE_EMPTY to make sure you only did the load if the table was empty, thus preventing adding the same data twice. This isn't directly supported in bq.py, but you could use the underlying bigquery_client.py to make this call, or use the REST api directly.