Looping througt ID's from database table in Pentaho Data Integration - pentaho

I am new to PDI and want to achieve looping. So the scenario goes like this...
I have a URL like {store}/admin/orders/{orderID}/metafields.json, So I have orderIDs in the table but I want to iteratively change the orderID in the URL.
For Example:
**OrderID** [1,2,3,.....n]
{store}/admin/orders/**1**/metafields.json
{store}/admin/orders/**2**/metafields.json
{store}/admin/orders/**3**/metafields.json
.
.
.
.
{store}/admin/orders/**n**/metafields.json
I tried to do it via variables but no success. It only returns the 1st orderID, not the rest.
The current setup is
**Transformation 1** ---> **table input** (orderIDs) --> **setVariables** (setting orderID as variable)
**Transforamtion 2** ---> **getVariables** (orderIDs) --> **textoutput** (for testing)
**job 1** ---> **start** --> **Transformation 1** (with **execute for every input row** option checked) --> **Transformation 2** --> **success**
any alternative will be highly appreciated. TIA

The way you are thinking for loop, is not correct in PDI. I have prepared a solution for you Loop, you can get help from here.
Please let me know if you need anything else.
Note: you need to run MainJob2.kjb.
Also, I have uploaded the output log file , after I run this job.

Related

SQL/BigQuery on github samples

I'm using the google bigquery tool, I'm trying to select ALL sample github repositories that have a pom.xml file and within the content of the file, have an artifact id ex-ex e.g
<artifactId>ex-ex</artifactId>
For this I have broken it down into 2 steps:
1) Find all pom.xml files
SELECT sample_repo_name FROM 'bigquery-public-data.github_repos.sample_contents' WHERE sample_path LIKE 'pom.xml'
2) Select the repositories which contain ex-ex artifact (in the content table)
AND content LIKE '%ex-ex'
The 2nd part of the query does not work (no results found) and is likely due to some syntax error somewhere. Full query below:
SELECT sample_repo_name FROM 'bigquery-public-data.github_repos.sample_contents' WHERE sample_path LIKE 'pom.xml' AND content LIKE '%ex-ex' LIMIT 1000
Would really appreciate help with this, thanks!
Have you tried '%ex-ex%'? Without the second %, you are only searching for records whose last 5 characters are 'ex-ex'. Adding content to the select in your first query and spot checking a few results, the content field appears to be XML (pom.xml, duh) and seem to end with </project>, and thus will probably never match with '%ex-ex'.

save entity-find results and iterate throught it later

I'm doing a conditional entity-find and want to save the results somewhere to iterate through it later. I searched around and found 2 solutions. I'm using mySql btw.
1-create a temp table and insert results into it
2-saving results in a file (csv, ...)
now my question is:
1-which solution is preferable or maybe suggest another solution (common/trusted pattern)
2- how to do it? (for example I don't know how to create a table on the fly or drop it in moqui.Suggesting a resource/example source code etc. would be awesome)
thanks in advance
One possible approach would be to convert the list that your entity-find gives you to binary data and store it as a DbResourceFile using the moqui.resource.DbResource entities, and using the utilities from org.apache.commons.io etc.
EDIT - To expound in response to comment below, I was thinking along such lines as
<set field="fileData" from="yourList.toString().getBytes()" />
<service-call name="create#moqui.resource.DbResource" in-map="[filename:'ExampleListFrom01012019.bin', isFile:'N']" out-map="context" />
<service-call name="create#moqui.resource.DbResourceFile" in-map="context + [mimeType:'application/octet-stream', fileData:fileData]" />
Then bring it back after a find on the DBResource with something like
<set field="convertedBack" from="x.fileData.getBinaryStream()" type="NewList" />
I haven't tried this, and there are no near samples in the code that I know of.
This type of conversion between types is not best practice, but storing lists to iterate through them later is probably not either.
Perhaps it would help if you elaborated on your business requirement.

Talend get current filename

I want to load excel files into the mysql database and check that they do not already exist, my problem is that I can not extract the name of the current file.
For example, I have the following files A.xlsx, B.xlsx and C.xls.
It return always B.xlsx
I think the issue you have is that your "RunIf" link is before the iterator, and therefore it's not being triggered at the right time.
The image below shows a simplified version, where I list the rows in the spreadsheets and then the file names. If I connected the second subjob to the tFileList_1 component, as you did in the question, I only get the logs from tLogRow_1 and the last filename by tLogRow_2.
With the link as shown, behind the iterator, then I get spreadsheet contents listed and then the title of it:
Col1|Col2|Col3
A|B|C
D|E|F
A.xlsx
Col1|Col2|Col3
A|B|C
D|E|F
B.xlsx
Col1|Col2|Col3
A|B|C
D|E|F
C.xlsx
I am assuming that you have filted out the duplicated files in the tMap component, so if you link your second subjob from the disponsibilite_3d component, I think you will get the result you are looking for.

Xero API > Invoices > PUT > Tracking Categories

Following:
https://developer.xero.com/documentation/api/invoices#post
It suggests you can add a Tracking Category when creating an invoice in the XML format:
<Tracking>
<TrackingCategoryID>ABC</TrackingCategoryID>
<Name>Name</TrackingCategoryID>
<Option>Option</Option>
</Tracking>
Within the
<LineItem>
node
Which is within the
<LineItems>
node
However, when doing this, the API returns:
Object reference not set to an instance of an object.
Which I understand to mean it doesn't recognise an element / node within the XML.
Without the node, everythings works fine, so it's the node and nodes within which cause the problem. Individually testing each sub node within results in the same problem.
I'm pretty sure I'm following the structure as per docs at https://developer.xero.com/documentation/api/invoices#post so if anyone can shed some light as to the correct structure to add Tracking Codes to an invoice line item...!
Most appreciated,
Thanks
Rob
I ran into the same issue. When posting XML for Tracking Options in Xero, the tracking data should be inside a TrackingCategory XML element, such as below:
<Tracking>
<TrackingCategory>
<TrackingCategoryID>ABC</TrackingCategoryID>
<Name>Name</Name>
<Option>Option</Option>
</TrackingCategory>
</Tracking>
TrackingCategoryID is optional, and you can have a maximum of 2 TrackingCategory elements.

Deleting a value from sql database using jsp

I'm trying to use jsp to rendering a simple database and have the ability to delete a row using a button. I'm running into the trouble of not being able to fetch the primary key from the row because I've been using to iterate through the table. I've tried passing just simple numbers to denote the primary key (integers) but its not grabbing it.
Lets say I want to delete the row with the primary key of 12.
<form action="deleteResponse.jsp">
<c:set var="numId" value="12"/>
<input type="submit" name="delete" value="Delete"/>
</form>
This generates a button which opens up deleteResponse.jsp which has the following to catch the value.
<sql:update var="counselorDelete" dataSource="jdbc/IFPWAFCAD">
DELETE FROM Counselor
WHERE counselor_id = ?<sql:param value="${param.numId}"/>
</sql:update>
It's not doing anything, and I'm not sure whats going on with it. If I manually put in the value 12 in the sql query it will delete the row but it won't do it if I try to fetch the value.
Thanks very much in advance!
Also as a side note. I'm having a hard time understanding how to properly write JSPs. I see that most of the time people use <% Java code %> but in the netbean tutorial, it uses the and functions. Could someone explain that to me? I'm also using glassfish as a localhost sql server. Netbean generates a glassfish xml but I also see people using DriverManager.getConnection methods. Which one is conventionally better?
I would try to separate the UI/JSP code from the DAO layer, using a seperate Bean to handle all the insert/uodate/delete operations.
Take a look at the logfiles of your Applicationserver, this might give you some hints.
Kr, R
Edit: Concerning the sql:
<sql:update var="counselorDelete" dataSource="jdbc/IFPWAFCAD"
sql="DELETE FROM Counselor WHERE counselor_id = ?" >
<sql:param value="${param.numId}"/>
</sql:update>