Return elements of a JSON object as an array - anypoint-studio

I have an XML file that is converted to a JSON object by a TRANSFORM MESSAGE:
%dw 2.0
output application/json
---
payload
The resulting JSON object has a format of:
{
"Items": {
"Item": {
"ItemId": "123",
"OrganizationId": "456",
"OrganizationCode": "ABC",
where there is one "Items" and 112 "Item". What I want is to return an JSON array contains all of the ItemIds. I am attempting to use TRANSFORM MESSAGE within a FOREACH. The FOREACH has 'payload' in the collection field and TRANSFORM MESSAGE has:
%dw 2.0
output application/json
---
myItems: payload.items.*item map (item, index) -> {
"myItemId" : item.ItemId
}
However, it always returns everything, the entire JSON object. I can't figure out if my FOREACH is wrong or if my TRANSFORM MESSAGE is wrong, but it always returns the entire JSON object.
An example of the incoming JSON would be:
{
"Items": {
"Item": {
"ItemId": "8041",
"OrganizationId": "12",
"OrganizationCode": "ABC",
},
"Item": {
"ItemId": "8050",
"OrganizationId": "12",
"OrganizationCode": "ABC",
},
"Item": {
"ItemId": "3801",
"OrganizationId": "12",
"OrganizationCode": "ABC",
}
}
}
The output should be: ["8041", "8050", "3801"]. Parse the Item elements, extract the ItemID value and create a JSON array.

Your script works nearly well. Allow me to mention that the first transformation to convert from XML to JSON is counterproductive. The next transformation will have to parse JSON to emit JSON. It is better to do it in one step, from XML to JSON. If you need the intermediate result for some other processing, it is better to convert to Java (application/java), process, then when the final output is needed, then convert to the final JSON format. Processing intermediate steps in XML or JSON just consumes more resources.
Based on that I used the following XML input:
<Items>
<Item>
<ItemId>8041</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
<Item>
<ItemId>8050</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
<Item>
<ItemId>3801</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
</Items>
Script:
%dw 2.0
output application/json
---
payload.Items.*Item map (item, index) -> item.ItemId
Output:
[
"8041",
"8050",
"3801"
]
The same script will work with your JSON input too.

Related

Need to change csv header from camel case to snake case in dataweave

I am working on a pipeline to dynamically dump all columns from the salesforce object to the S3 bucket.
I get all columns for a salesforce object using describe object API. I store all columns into a variable and then create a big SOQL query out of it and submit a bulk query job v2.
Now, this is the main problem. The Column name I am getting from the salesforce connector is in camelCase
[{
"Id": 123,
"FirstName": "Manual",
"MasterRecordId__c" :"abc"
},
{
"Id": 456,
"FirstName": "John",
"MasterRecordId__c" :"def"
}]
But I want column names to be in snake case
[{
"Id": 123,
"first_name": "Manual",
"master_record_id__c":"abc"
},
{
"Id": 456,
"first_name": "john",
"master_record_id__c":"def"
}]
I understand mulesoft has an underscore function to do the same thing, but I am not able to apply any function at "key" level.
Any lead would be really helpful. Please let me know for any questions.
You just have to use mapObject along with the underscore function
%dw 2.0
import underscore from dw::core::Strings
output application/json
---
payload map ((item) ->
item mapObject ((value, key) -> {
(underscore(key)): value
})
)
In case you want Id field to remain as it is, give a try like below:
%dw 2.0
import * from dw::core::Strings
output application/json
---
payload map ($ mapObject ((value, key, index) -> if (capitalize(key as String) == key as String)
{
(key): value
}
else
{
(underscore(key)): value
}))

Dataweave - how to accumulate json response from http request connector in for each component (Mule 4)

I consume a HTTP request and I need to save and accumulate the JSON response with out any transformation in a variable, I do that but I dont know why the accumulate it correctly, could you please tell me how can I solve that problem.
Json Response By Iteration:
Iteration 1:
{
"orderId": "11111",
"status": "false",
"receivedAt": "2022-07-28T22:45:12.175Z",
"createdAt": "2022-07-28T22:45:12.175Z",
}
Iteration 2:
{
"orderId": "22222",
"status": "false",
"receivedAt": "2022-07-28T22:45:27.907Z",
"createdAt": "2022-07-28T22:45:27.907Z"
}
Dataweave: (csv Payload is the name of the variable where the values accumulated)
%dw 2.0
output application/json
---
if ( vars.counter == 1)
( payload )
else
( vars.csvPayload ) ++ payload
Variable Result:
{
"orderId": "11111",
"status": "false",
"receivedAt": "2022-07-28T22:45:12.175Z",
"createdAt": "2022-07-28T22:45:12.175Z",
"orderId": "22222",
"status": "false",
"receivedAt": "2022-07-28T22:45:27.907Z",
"createdAt": "2022-07-28T22:45:27.907Z"
}
Variable Expected:
[
{
"orderId": "11111",
"status": "false",
"receivedAt": "2022-07-28T22:45:12.175Z",
"createdAt": "2022-07-28T22:45:12.175Z",
},
{"orderId": "22222",
"status": "false",
"receivedAt": "2022-07-28T22:45:27.907Z",
"createdAt": "2022-07-28T22:45:27.907Z"
}
]
NOTE: I don't know why the JSON responses of each iteration are joined in the same object and not as a different object in an array.
The incorrect output is easy. You are concatenating an object to another object. In that case the operator ++ "extracts all the key-values pairs from each object, then combines them together into one result object." according to the documentation. The parenthesis are totally unneeded.
Setting the output to JSON in each iteration -I'm assuming this is inside a foreach- is inefficient since it requires formatting the output to JSON in each iteration only to parse it again in the following one. I recommend to use application/java -which doesn't require parsing/formatting- inside a loop and after the loop convert the entire output to JSON in one go.
You should use an array to hold the values, so assign an empty array ([]) to the variable before the foreach loop to initialize it. Then the counter is unneeded since you can just add elements to the array:
%dw 2.0
output application/java
---
vars.allOrders ++ payload
Then after the foreach just transform the array to JSON:
%dw 2.0
output application/json
---
vars.allOrders
Your requirement is a very common scenario in integrations.
Assuming you are using HTTP Requester inside a for-each loop and initialized the variable to hold the final Payload (lets say var finalPayload = []) before the loop, you have to keep on updating the same variable (append the new data in the same array variable like below) with the response payload received from HTTP Requester for each iteration.
var finalPayload << payload
Ultimately check the finalPayload outside for-each loop, that will give you your desired result.

dataweave script to iterate over a nested array of string literals

So I have the following payload :
<?xml version="1.0" encoding="utf8" ?>
<Output>
<Error>
<Status>0</Status>
<Details>No errors</Details>
</Error>
<Synopsis>
<Count>451</Count>
</Synopsis>
<BankAccounts>
<BankAccount AcctNo="103" CustName="Frank" BalanceAmount="" Inactive="N" NoOfAccounts="1" >
<Addresses>
<Address>ABC</Address>
<Address>XYZ</Address>
</Addresses>
</BankAccount>
<BankAccount AcctNo="101" CustName="Jane" BalanceAmount="10005" Inactive="N" NoOfAccounts="1" >
<Addresses>
<Address>LMN</Address>
<Address>QWE</Address>
</Addresses>
</BankAccount>
</BankAccounts>
</Output>
I need to convert it into the following json :
[
{
"accountNumber": "103",
"customerName": "Frank",
"balanceAmount": "",
"inactive": "N",
"noOfAccounts": "1",
"address": [
"ABC","XYZ"
]
},
{
"accountNumber": "101",
"customerName": "Jane",
"balanceAmount": "10005",
"inactive": "N",
"noOfAccounts": "1",
"address": [
"LMN","QWE"
]
}
]
I am nearly there using the below dataweave script:
%dw 2.0
output application/json writeAttributes=true
---
payload.Output.BankAccounts.*BankAccount map {
accountNumber: $.#AcctNo,
customerName: $.#CustName,
balanceAmount: $.#BalanceAmount,
inactive: $.#Inactive,
noOfAccounts: $.#NoOfAccounts,
address: $.Addresses.*Address map (value,index)->{
}
}
However I have not been able to convert the Addresses element ( which contains multiple Address elements into a string array like this :
"address": [
"LMN","QWE"
]
Am struggling with the last part , any ideas ?
Note that XML does not has the concept of an array. Addresses is just an element that has several address child elements.
You can use the multivalued selector directly. It will return an array of the matching keys values. Map is only needed if you want to further transform each element.
$.Addresses.*Address
You can use the value from the address array without using {}. DW creates an object when curly braces are used.
%dw 2.0
output application/json
---
payload.Output.BankAccounts.*BankAccount map ((account) -> {
accountNumber: account.#AcctNo,
customerName: account.#CustName,
balanceAmount: account.#BalanceAmount,
inactive: account.#Inactive,
noOfAccounts: account.#NoOfAccounts,
address: account.Addresses.*Address map ((value)-> value)
})

How to extract the field from JSON object with QueryRecord

I have been struggling with this problem for a long time. I need to create a new JSON flowfile using QueryRecord by taking an array (field ref) from input JSON field refs and skip the object field as shown in example below:
Input JSON flowfile
{
"name": "name1",
"desc": "full1",
"refs": {
"ref": [
{
"source": "source1",
"url": "url1"
},
{
"source": "source2",
"url": "url2"
}
]
}
}
QueryRecord configuration
JSONTreeReader setup as Infer Schema and JSONRecordSetWriter
select name, description, (array[rpath(refs, '//ref[*]')]) as sources from flowfile
Output JSON (need)
{
"name": "name1",
"desc": "full1",
"references": [
{
"source": "source1",
"url": "url1"
},
{
"source": "source2",
"url": "url2"
}
]
}
But got error:
QueryRecord Failed to write MapRecord[{references=[Ljava.lang.Object;#27fd935f, description=full1, name=name1}] with schema ["name" : "STRING", "description" : "STRING", "references" : "ARRAY[STRING]"] as a JSON Object due to java.lang.ClassCastException: null
Try the following approach, in your case it shoud work:
1) Read your JSON field fully (I imitated it with GenerateFlowFile processor with your example)
2) Add EvaluateJsonPath processor which will put 2 header fileds (name, desc) into the attributes:
3) Add SplitJson processor which will split your JSON byt refs/ref/ groups (split by "$.refs.ref"):
4) Add ReplaceText processor which will add you header fields (name, desc) to the split lines (replace "[{]" value with "{"name":"${json.name}","desc":"${json.desc}","):
5) It`s done:
Full process in my demo case:
Hope this helps.
Solution!: use JoltTransformJSON to transform JSON by Jolt specification. About this specification.

How to get unnamed object instead of an array in case of only one result in Dataweave

We have a transformation with Dataweave which processes a list of objects. We get a json response like that:
{"hotels": [{
"name": "Hotel Oslo",
"propertyCode": "12345",
"currency": "NOK"
},
{
"name": "Hotel Stockholm",
"propertyCode": "12346",
"currency": "SEK"
}]}
However, in the case of only 1 response, we want to have the following response:
{"name": "Hotel Stockholm",
"propertyCode": "12346",
"currency": "SEK"}
We are generating the response like this:
{
hotels: payload.rows map ((row , indexOfRow) -> {
name: row.content.companyName.content,
propertyCode: row.content.propertyCode.content,
currency: row.content.currencyCode.content
})}
What should we put as a condition so that we do not get an array in case of 1 result?
Try this:
%dw 1.0
%output application/json
%function makeHotel(row) {
name: row.name,
propertyCode: row.propertyCode,
currency: row.currency
}
---
{
hotels: payload.rows map ((row , indexOfRow) -> makeHotel(row))
} when ((sizeOf payload.rows) != 1)
otherwise makeHotel(payload.rows[0])
It will give you an empty array on empty input, the simple object for one input and the structure with array when you have more than one input row.
(For test purposes, with a slightly differnt input structure, but the general solution should be clear.)