I am working with Talend Open Studio for Data Integration.
I want to create a simple job which shows all customers from database with specific city.
My job structure looks like this:
DbConnection -- onComponentOk -- DbInput -- row1-- tJavaRow -- row2 -- tLogRow
I have created a context parameter that contains specific values which are the city ids. I want to set the city manually after the job starts, and then pass it to my query on the WHERE clause. Is it possible to do this scenario with Talend? How should my tJavaRow code should look like?
If you want to manually input something in a running job, you can use a tMsgBox. In Component, set buttons -> Question, the rest of the settings depends on you.
You will be able to input a value. That value is retrievable from the variable RESULT of the component.
Example with tMsgBox_1
(String)globalMap.get("tMsgBox_1_RESULT")
Related
So from last few weeks I was trying to design a SSIS package that would read some XML files that I have and move the data from it to the multiple tables I want.
These file contains different nodes like Individual (parent node) and Address, Alias, Articles (all child nodes of Individual) etc.
Data in those files look like this:
<Individuals>
<Individual>
<UniqueID>1001</UniqueID>
<Name>Ben</Name>
<Soft_Delete>N</Soft_Delete>
<Soft_Delete_Date>NULL</Soft_Delete_Date>
</Individual>
<Addresses>
<Address>
<Address_Line_1>House no 280</Address_Line_1>
<Address_Line_2>NY</Address_Line_2>
<Country>US</Country>
<Soft_Delete>N</Soft_Delete>
<Soft_Delete_Date>NULL</Soft_Delete_Date>
</Address>
<Address>
<Address_Line_1>street 100</Address_Line_1>
<Address_Line_2>California</Address_Line_2>
<Country>US</Country>
<Soft_Delete>N</Soft_Delete>
<Soft_Delete_Date>NULL</Soft_Delete_Date>
</Address>
</Addresses>
</Individuals>
I was successful in designing it and now I have a different task.
The files I had were named like this: Individual_1.xml,Individual_2.xml,Individual_3.xml etc.
Now I have received some new files which are named like this:
Individual_UPDATE_20220716.xml,Individual_UPDATE_20220717.xml,Individual_UPDATE_20220718.xml,Individual_UPDATE_20220720.xml etc
Basically these files contains the updated information of previously inserted records
OR
There are totally new records
For example:
A record or a particular information like Address of an Individual was Soft Deleted.
Now I am wondering how would I design or modify my current SSIS package to update the data from these new files into my database?
Any guidance would be appreciated....
Thank you...
It looks like you have no problem reading the XML so I won't really talk about that. #Yitzack's comment to his prior answer is "a" way to do it. However, his answer assumes you can create staging tables. To do this entirely inside SSIS, the way to do it is as follows...
I would treat all the files the same (as long as they have the same data structure and it seems like the case.
Read the XML as a source.
Proceed to a lookup.
Set the lookup to ignore errors (this is handled in the next step)
Set your lookup to the destination table and lookup uniqueID and add to the data flow. Since you said ignore errors, SSIS will insert a null in that field if the lookup fails to find a match.
Condition Split based on the destination.UniqueID == null and call that output inserts and change the default to updates.
Add a SQL statement to update the existing record and map your row to it (this is somewhat slow and why the merge is better with large data sets).
Connect from Condition Split to update SQL statement and map appropriately.
Add an insert, connect the proper output from Conditional Split and map.
Note: It looks like you are processing from a file system, and it is likely that order is important. You may have to order you foreach loop. I will provide a simple example that you can modify.
Create a filesInOrder variable of type object.
Create a script task.
Add filesInOrder as a read/write variable
Enter script...
var diFiles = new System.IO.DirectoryInfo(#"path to your folder").GetFiles("*.xml");
var files = diFiles.OrderBy(o => o.CreationTime).Select(s => s.FullName);
Dts.Variables["filesInOrder"].Value = files.ToArray();
Make sure you add Using System.Linq; to your code.
Finally, use filesInOrder as your base to a foreach component based on an ADO enumerator (filesInOrder).
How to handle the remove flight booking in webtours? It came to my mind that if I wanna run the test with 3 virtual users and how they supposed to delete the booking if the information down here (refer below) are unique for each virtual user? Does any of these variables below need to be parameterized or need to apply correlation?
If there are multiple flight id to be deleted, we can create a CSV file having those ids
and use a while loop controller to delete each of them.
You may refer -
https://guide.blazemeter.com/hc/en-us/articles/206733689-Using-CSV-DATA-SET-CONFIG
https://www.blazemeter.com/blog/using-while-controller-jmeter
As you said, it will differ for a different user, you can store each value from the response of each user(extract value using regex) and pass them in delete call.
reference link - https://guide.blazemeter.com/hc/en-us/articles/207421325-Using-RegEx-Regular-Expression-Extractor-with-JMeter-Using-RegEx-(Regular-Expression-Extractor)-with-JMeter
I don't know what "webtours" is, however there is one "golden" rule: each and every parameter which is dynamic needs to be correlated.
If you don't have any idea regarding which ones are dynamic - record the same action one more time using JMeter's HTTP(S) Test Script Recorder and compare the generated requests, parameter values (or even names) which differ needs to be correlated.
Another approach is to inspect the previous response(s) using View Results Tree listener and look if the data is present there
Check out Advanced Load Testing Scenarios with JMeter: Part 1 - Correlations article for more information and example implementation
Apologies for long message. I am new to mule ESB and trying to elaborate my issue.
My requirment is
Get the input in URL as query parameter like college name.
Connect database and do select query using "college name" in where condition.
Get the result set(example - returns multiple department in the college with metadata) and iterate result set under for each loop.
a) Process the first row and construct the XML and add http headers.
b) hit the vendor service.
c) Receive response from vendor service.
d) Process the vendor service -- Kind of manipulating fees in the response.
c) Set the manupulated fees and against department in the array variable.
d) Remove the processed "rootMessage" to free up memory.
After processing all the five rows (limited 5 row selection in the select command). Now we will have five sets of variables. Each variable should have dept and fees.
Run a update command in database with "dept" and "fees".
To achieve above requirement, I have built below specified mule flow.
Here I am able to complete till step 3. I got struck in step 4 and step 5.
My query is
how to set the values in array kind of variable in mule ESB /Data weave in the scope of 'for Each'. It should be incremented. My case it is keep overwritten by the next row/message. Since i could not create variable to increment itself.
Please show some light on this.
Thanks in advance!
In the editor you can set a flow variable to store a list.
It should be initialized
flowVars.myList = []
then added to on each loop
flowVars.myList.add(element);flowVars.myLis
I have a problem creating sql dynamic action in oracle apex v4.2. I have two fields, Department number and department name. Department number is a text field with autocomplete. The department name is a display field. On changing the department number, the department name should be displayed by an sql query.
I created a set value dynamic action on department number, giving the correct values in page item to submit and the correct sql query referencing P3_DEPARTMENT_NO.
When i run the page, after select a department number, the department name is not coming up automatically.
Could you please suggest on what i might be missing.
Thanks in advance.
You can try use this way:
First step: In the Shared Components -> Application Processes: create a process myProcess an put your sql dynamic in.
Second step: Create a javascript function myFunction to call the process myProcess.
Third step: Use onChange event to call your javascript function myFunction.
Also you can find a lot of exemples on Denes Kubicek app: https://apex.oracle.com/pls/otn/f?p=31517:101:116042570427567.
Best regards,
iulian
The exact behaviour of autocomplete lists is probably browser dependent, but generally speaking, don't rely on the "Change" event, as it won't necessarily fire when you select from the list.
You'll need to experiment to get the behaviour you want in your particular situation, but as a starting point you might want to try replacing the "Change" event type on your dynamic action with "Lose focus". That way the dynamic action should always be triggered when you tab or click away from P3_DEPARTMENT_NO.
In similar situations in the past, I've used "Key release" instead of "Lose focus", and I've created a second dynamic action which does the same thing, but triggered by "Get focus". That combination ensures that the display field stays synchronised with the user's selection, whether a value is keyed in or selected from the autocomplete list. Whether or not you go this route depends on how happy you are about the database being hit with your department name query every time a user interacts with P3_DEPARTMENT_NO in any way.
Hi does any one know how to check if a given DB object (Table/View/SP/Function) is used inside Oracle.
For example to check if the table "A" is used in any SP/Function or View definitions. I am trying to cleanup unused objects in the database.
I tried the query select * from all_source WHERE TEXT like '%A%' (A is the table name). Do you thing it is safe to assume it is not being used if it does not return any results?
From this ASKTOM question:
You'll have to enable auditing and then come back in 3 months to see.
We don't track this information by default -- also, even with auditing, it may be very
possible to have an object that is INDIRECTLY accessed (eg: via a foreign key for
example) that won't show up.
You can try USER_DEPENDENCIES but that won't tell you about objects referenced by code in
client apps or via dynamic sql
There's code in the thread for checking ALL_SOURCE, but it's highlighted that this isn't a silver bullet.