I am using Mule 4.3 and Dataweave 2.x
From database am getting following records example:
[
{
"gp": "G1",
"parent": "P1",
"child": "C1"
},
{
"gp": "G1",
"parent": "P1",
"child": "C2"
},
{
"gp": "G2",
"parent": "P1",
"child": "C3"
},
{
"gp": "G1",
"parent": "G1",
"child": "C4"
}
]
Each element in array represents a row of data and each row has hierarchical data (Grandparent , Parent and Child data)
I need to generate the output in XML as below:
<list>
<genealogy>
<code>G1</code>
<hierarchy>
<!-- here parent is not included in hierarchy as it has same value of grandparent -->
<genealogy>
<code>C4</code>
<hierarchy/>
</genealogy>
<!-- here 2 instances of P1 have become One value -- >
<genealogy>
<code>P1</code>
<hierarchy>
<genealogy>
<code>C1</code>
<hierarchy/>
</genealogy>
<genealogy>
<code>C2</code>
<hierarchy/>
</genealogy>
</hierarchy>
</genealogy>
</hierarchy>
</genealogy>
<genealogy>
<code>G2</code>
<hierarchy>
<genealogy>
<code>P1</code>
<hierarchy>
<genealogy>
<code>C3</code>
<hierarchy/>
</genealogy>
</hierarchy>
</genealogy>
</hierarchy>
</genealogy>
</list>
Here couple of challenges are:
Its a 3 level Hierarchy where Grandparent needs to be the first element , within which we have the Parent(s) and within Parent Child(ren)
If the parent ( Ex : G1 last object in input array ) has same value of grandparent then parent should be dropped in hierarchy ( Grandparent and then directly Child )
I am stumped how to do the needful in Dataweave and have tried things like groupBy , pluck etc but am not able to get desired result
I have built a solution. Somehow, making this generic was more easier then make it specifically for 3 levels.
It is a bit complicated, I have explained the logic in comments within the script
%dw 2.0
output application/xml
// hierarchy's structure. If, lets say you add a "grand child" it should work. (haven't tested this though)
var hierarchy = {
gp: {
parent: {
child: null
}
}
}
fun getDirectGeanologies(records, hierarchyStructure) = do {
var topLevel = keysOf(hierarchyStructure)[0] as String
var secondLevel = keysOf(hierarchyStructure[topLevel])[0]
---
(records groupBy $[topLevel])
mapObject ((level1Childs, code) ->
genealogy: {
code: code,
hierarchy:
if(secondLevel != null) // If there are no more childs, do nothing.
(level1Childs groupBy ((item) -> item[secondLevel] ~= code) // This is to handle when parent = gp
mapObject ((value, hasSameParent) ->
// If parent = gp, we want to skip parent and link directly to child
if(hasSameParent as Boolean) getDirectGeanologies(value, hierarchyStructure[topLevel][secondLevel])
// recurrsively call the function with child records and going down the hierarchyStructure by one
else getDirectGeanologies(value, hierarchyStructure[topLevel])
))
else {}
}
)
}
---
list: getDirectGeanologies(payload,hierarchy)
Related
I am working on a pipeline to dynamically dump all columns from the salesforce object to the S3 bucket.
I get all columns for a salesforce object using describe object API. I store all columns into a variable and then create a big SOQL query out of it and submit a bulk query job v2.
Now, this is the main problem. The Column name I am getting from the salesforce connector is in camelCase
[{
"Id": 123,
"FirstName": "Manual",
"MasterRecordId__c" :"abc"
},
{
"Id": 456,
"FirstName": "John",
"MasterRecordId__c" :"def"
}]
But I want column names to be in snake case
[{
"Id": 123,
"first_name": "Manual",
"master_record_id__c":"abc"
},
{
"Id": 456,
"first_name": "john",
"master_record_id__c":"def"
}]
I understand mulesoft has an underscore function to do the same thing, but I am not able to apply any function at "key" level.
Any lead would be really helpful. Please let me know for any questions.
You just have to use mapObject along with the underscore function
%dw 2.0
import underscore from dw::core::Strings
output application/json
---
payload map ((item) ->
item mapObject ((value, key) -> {
(underscore(key)): value
})
)
In case you want Id field to remain as it is, give a try like below:
%dw 2.0
import * from dw::core::Strings
output application/json
---
payload map ($ mapObject ((value, key, index) -> if (capitalize(key as String) == key as String)
{
(key): value
}
else
{
(underscore(key)): value
}))
So I have the following payload :
<?xml version="1.0" encoding="utf8" ?>
<Output>
<Error>
<Status>0</Status>
<Details>No errors</Details>
</Error>
<Synopsis>
<Count>451</Count>
</Synopsis>
<BankAccounts>
<BankAccount AcctNo="103" CustName="Frank" BalanceAmount="" Inactive="N" NoOfAccounts="1" >
<Addresses>
<Address>ABC</Address>
<Address>XYZ</Address>
</Addresses>
</BankAccount>
<BankAccount AcctNo="101" CustName="Jane" BalanceAmount="10005" Inactive="N" NoOfAccounts="1" >
<Addresses>
<Address>LMN</Address>
<Address>QWE</Address>
</Addresses>
</BankAccount>
</BankAccounts>
</Output>
I need to convert it into the following json :
[
{
"accountNumber": "103",
"customerName": "Frank",
"balanceAmount": "",
"inactive": "N",
"noOfAccounts": "1",
"address": [
"ABC","XYZ"
]
},
{
"accountNumber": "101",
"customerName": "Jane",
"balanceAmount": "10005",
"inactive": "N",
"noOfAccounts": "1",
"address": [
"LMN","QWE"
]
}
]
I am nearly there using the below dataweave script:
%dw 2.0
output application/json writeAttributes=true
---
payload.Output.BankAccounts.*BankAccount map {
accountNumber: $.#AcctNo,
customerName: $.#CustName,
balanceAmount: $.#BalanceAmount,
inactive: $.#Inactive,
noOfAccounts: $.#NoOfAccounts,
address: $.Addresses.*Address map (value,index)->{
}
}
However I have not been able to convert the Addresses element ( which contains multiple Address elements into a string array like this :
"address": [
"LMN","QWE"
]
Am struggling with the last part , any ideas ?
Note that XML does not has the concept of an array. Addresses is just an element that has several address child elements.
You can use the multivalued selector directly. It will return an array of the matching keys values. Map is only needed if you want to further transform each element.
$.Addresses.*Address
You can use the value from the address array without using {}. DW creates an object when curly braces are used.
%dw 2.0
output application/json
---
payload.Output.BankAccounts.*BankAccount map ((account) -> {
accountNumber: account.#AcctNo,
customerName: account.#CustName,
balanceAmount: account.#BalanceAmount,
inactive: account.#Inactive,
noOfAccounts: account.#NoOfAccounts,
address: account.Addresses.*Address map ((value)-> value)
})
I have an XML file that is converted to a JSON object by a TRANSFORM MESSAGE:
%dw 2.0
output application/json
---
payload
The resulting JSON object has a format of:
{
"Items": {
"Item": {
"ItemId": "123",
"OrganizationId": "456",
"OrganizationCode": "ABC",
where there is one "Items" and 112 "Item". What I want is to return an JSON array contains all of the ItemIds. I am attempting to use TRANSFORM MESSAGE within a FOREACH. The FOREACH has 'payload' in the collection field and TRANSFORM MESSAGE has:
%dw 2.0
output application/json
---
myItems: payload.items.*item map (item, index) -> {
"myItemId" : item.ItemId
}
However, it always returns everything, the entire JSON object. I can't figure out if my FOREACH is wrong or if my TRANSFORM MESSAGE is wrong, but it always returns the entire JSON object.
An example of the incoming JSON would be:
{
"Items": {
"Item": {
"ItemId": "8041",
"OrganizationId": "12",
"OrganizationCode": "ABC",
},
"Item": {
"ItemId": "8050",
"OrganizationId": "12",
"OrganizationCode": "ABC",
},
"Item": {
"ItemId": "3801",
"OrganizationId": "12",
"OrganizationCode": "ABC",
}
}
}
The output should be: ["8041", "8050", "3801"]. Parse the Item elements, extract the ItemID value and create a JSON array.
Your script works nearly well. Allow me to mention that the first transformation to convert from XML to JSON is counterproductive. The next transformation will have to parse JSON to emit JSON. It is better to do it in one step, from XML to JSON. If you need the intermediate result for some other processing, it is better to convert to Java (application/java), process, then when the final output is needed, then convert to the final JSON format. Processing intermediate steps in XML or JSON just consumes more resources.
Based on that I used the following XML input:
<Items>
<Item>
<ItemId>8041</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
<Item>
<ItemId>8050</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
<Item>
<ItemId>3801</ItemId>
<OrganizationId>12</OrganizationId>
<OrganizationCode>ABC</OrganizationCode>
</Item>
</Items>
Script:
%dw 2.0
output application/json
---
payload.Items.*Item map (item, index) -> item.ItemId
Output:
[
"8041",
"8050",
"3801"
]
The same script will work with your JSON input too.
I have followed the doc for the ReferenceArrayInput (https://marmelab.com/react-admin/Inputs.html#common-input-props) but it does not seem to be working with relationship fields.
For example, I have this many-to-many relation for my Users (serialized version) :
Coming from (raw response from my API):
I have setup the ReferenceArrayInput as followed :
<ReferenceArrayInput source="profiles" reference="profiles" >
<SelectArrayInput optionText="label" />
</ReferenceArrayInput>
I think it's making the appropriate calls :
But here is my result :
Any idea what I'm doing wrong ?
Thanks in advance for your help !
On docs, ReferenceArrayInput is said to expect a source prop pointing to an array os ids, array of primitive types, and not array of objects with id. Looks like you are already transforming your raw response from api, so if you could transform a bit more, mapping [{id}] to [id], it could work.
If other parts of your app expects profiles to be an array of objects, just create a new object entry like profilesIds or _profiles.
As gstvg said, ReferenceArrayInput expects an array of primitive type, not array of objects.
If your current record is like below:
{
"id": 1,
"tags": [
{ id: 'programming', name: 'Programming' },
{ id: 'lifestyle', name: 'Lifestyle' }
]
}
And you have a resource /tags, which returns all tags like:
[
{ id: 'programming', name: 'Programming' },
{ id: 'lifestyle', name: 'Lifestyle' },
{ id: 'photography', name: 'Photography' }
]
Then you can do something like this (it will select the tags of current record)
<ReferenceArrayInput
reference="tags"
source="tags"
parse={(value) => value && value.map((v) => ({ id: v }))}
format={(value) => value && value.map((v) => v.id)}
>
<AutocompleteArrayInput />
</ReferenceArrayInput>
Here is my xml, in that first I need to check 'RecordsEntries' should not be 'null', then RecordEntry shouldn't be null followed by mapping code
<?xml version="1.0" encoding="UTF-8"?>
<Records>
<storenumber />
<calculated>false</calculated>
<subTotal>12</subTotal>
<RecordsEntries>
<RecordEntry>
<deliverycharge>30.0</deliverycharge>
<entryNumber>8</entryNumber>
<Value>true</Value>
</RecordEntry>
<RecordEntry>
<deliverycharge>20.0</deliverycharge>
<entryNumber>7</entryNumber>
<Value>false</Value>
</RecordEntry>
</RecordsEntries>
<RecordsEntries>
<RecordEntry>
<deliverycharge>30.0</deliverycharge>
<entryNumber>8</entryNumber>
<Value>false</Value>
</RecordEntry>
</RecordsEntries>
</Records>
Tried multiple scenario's also used when condition checking but somewhere missing parenthesis or correct format
orders: {
order: {
StoreID: payload.Records.storenumber,
Total: payload.Records.calculated,
(( payload.Records.RecordsEntries.*RecordEntry ) map {
IndividualEntry: {
Number:$.entryNumber,
DeliverCharge:$.deliverycharge
}
} when ( payload.Records.RecordsEntries != null and payload.Records.RecordsEntries.*RecordEntry !=null))}
}
getting error like missing ). Tried other way around by checking the null condition directly inside the first loop got error like "Cannot coerce array to an boolean". Please suggest. Thanks.
You can instead do
%dw 1.0
%output application/xml
---
orders: {
order: {
StoreID: payload.Records.storenumber,
Total: payload.calculated,
((payload.Records.*RecordsEntries.*RecordEntry default []) map {
IndividualEntry: {
Number:$.entryNumber,
DeliverCharge:$.deliverycharge
}
})
}
}
DataWeave is "null-safe" for querying values like in payload.Records.*RecordsEntries.*RecordEntry, but an error will be thrown when trying to operate with a null (e.g. null map {}).
The default operator replaces the value to its left, if it's null, with the one on the right.
Also you were missing an *. You need to use it in xml whenever you want all the repetitive elements that match.
If you want to skip a particular field assignment in dataweave based on the null or blank , below mentioned script can be used.
payload map ((payload01 , indexOfPayload01) -> { Name: payload01.Name, (LastName: payload01.LastName) when payload01.LastName !=null and payload01.LastName !='' })
This will skip the field mapping for Name , if the name field is coming as null or blank in the incoming payload.