Dynamic CSV with Header only once - mule

I have a request to parse a JSON payload and then create columns dynamically based on a condition.
There should be a Groups column with header. For any additional groups the employee is in, they will be in a column with no header.
If the member is in one group it makes sense I can do something like the following:
%dw 2.0
output application/csv separator=","
var employeesPayload = payload
---
employeesPayload filter ($.workEmail != '' and $.companyEmploymentType.isContractor == false) map (employee) -> {
"Employee ID": employee.employeeNumber,
"Name": employee.preferredFirstName default employee.firstName ++ ' ' ++ if (employee.preferredLastName == '' or employee.preferredLastName == null) employee.lastName else employee.preferredLastName,
"Email": employee.workEmail,
"Groups": employee.workState
}
i.e the table should look similar to the following:
But, how do I add additional columns without headers?
i.e if I want to add a user like Tito (row 9) in the screenshot, how can I build this dynamically?

You can add additional fields dynamically by mapping the payload. If you want the header to be empty you can set the key to an empty string. Note that you can not skip columns, if there is no content you need to at least output an empty string.
Example:
%dw 2.0
output application/csv
---
payload map {
($), // just reusing the input payload as is
d: $$, // some calculated field with a header
"": if (isEven($$)) $$ else "", // calculated field with an empty name, only on some condition
"": $$ // another calculated field with an empty name
}
Input:
[
{
"a": "a1",
"b": "b1",
"c": "c1"
},
{
"a": "a2",
"b": "b2",
"c": "c2"
},
{
"a": "a3",
"b": "b3",
"c": "c3"
}
]
Output:
a,b,c,d,,
a1,b1,c1,0,0,0
a2,b2,c2,1,,1
a3,b3,c3,2,2,2

Related

Need to change csv header from camel case to snake case in dataweave

I am working on a pipeline to dynamically dump all columns from the salesforce object to the S3 bucket.
I get all columns for a salesforce object using describe object API. I store all columns into a variable and then create a big SOQL query out of it and submit a bulk query job v2.
Now, this is the main problem. The Column name I am getting from the salesforce connector is in camelCase
[{
"Id": 123,
"FirstName": "Manual",
"MasterRecordId__c" :"abc"
},
{
"Id": 456,
"FirstName": "John",
"MasterRecordId__c" :"def"
}]
But I want column names to be in snake case
[{
"Id": 123,
"first_name": "Manual",
"master_record_id__c":"abc"
},
{
"Id": 456,
"first_name": "john",
"master_record_id__c":"def"
}]
I understand mulesoft has an underscore function to do the same thing, but I am not able to apply any function at "key" level.
Any lead would be really helpful. Please let me know for any questions.
You just have to use mapObject along with the underscore function
%dw 2.0
import underscore from dw::core::Strings
output application/json
---
payload map ((item) ->
item mapObject ((value, key) -> {
(underscore(key)): value
})
)
In case you want Id field to remain as it is, give a try like below:
%dw 2.0
import * from dw::core::Strings
output application/json
---
payload map ($ mapObject ((value, key, index) -> if (capitalize(key as String) == key as String)
{
(key): value
}
else
{
(underscore(key)): value
}))

Matching dynamic keys in karate

I am trying to match a JSON array with a predefined expected json. The problem is that one of the key values in actual JSON is a set of strings delimited by "|". Here is how it looks :
actualJSON = [
{
"a": "varA",
"mix: "X|Y|Z",
},
{
"b": "B",
"c": "C"
} ]
expectedJSON = [
{
"a": "varA",
"mix: "Y|Z|X",
},
{
"b": "B",
"c": "C"
} ]
Issue here is the mix key that represents a set of strings and the value can be any combination of "X|Y|Z" without any specific order like "Z|Y|X" etc. When the value of mix is Y|Z|X then
* match actualJSON contains expectedJSON
works fine but in other cases it fails as expected. Is there a way to do the matching when key value is dynamic?
My first suggestion is as far as possible to write tests where the response is 100% predictable and don't waste time on these weird cases. Also refer: https://stackoverflow.com/a/50350442/143475
That said this is easy to do if you write a JS function:
* def response = { foo: 'X|Y|Z' }
* def hasXyz = function(x){ return x.includes('X') && x.includes('Y') && x.includes('Z') }
* match response == { foo: '#? hasXyz(_)' }
I leave it to you to figure out better logic if you want. Refer: https://github.com/karatelabs/karate#self-validation-expressions

Splitting concatenated values in table column using Dataweave

I have sample data in my Salesforce table as given below:
I am on Mule 3.9 and running dataweave 1.0.
I need to use dataweave to read the above data (from a Salesforce table) and transform it into a JSON as given below:
[
{"Id": "634594cc","Name": "Alpha","List": "AB01"},
{"Id": "634594cc","Name": "Alpha","List": "AB02"},
{"Id": "634594cc","Name": "Alpha","List": "AB03"},
{"Id": "5d839e9c","Name": "Bravo","List": "CD01"},
{"Id": "5d839e9c","Name": "Bravo","List": "CD02"},
{"Id": "3a5f34d3","Name": "Charlie","List": null}
]
As you can see above, the "List" column is what I need to split as separate arrays in the final JSON. It has data with semicolon at the begin, in between and in the end.
Thanks in advance for your help.
I took the liberty to create a couple of sample data based upon the SS (BTW, its best not to use SS) :).
Try this:
%dw 1.0
%output application/dw
%var data = [
{
id: "ABC123",
name: "A",
list: ";AB1;AB2;AB3;"
},
{
id: "ZXY321",
name: "B",
list: null
}
]
---
data reduce (e,result=[]) -> (
result ++ using (
list = e.list default "" splitBy /;/ filter ($ != ""),
sharedFields = {
Id: e.id,
Name: e.name
}
) (
using (
flist = list when ((sizeOf list) > 0) otherwise [null]
) (
flist map {
(sharedFields),
List: $
}
)
)
)

Filter Payload with Dataweave for a Set Variable component

For the Mulesoft 4.2 Set Variable component, I want to assign a simple String value pulled from a single specific record from an incoming JSON payload.
In my case, taking just the value from the 'country' field from below example:
[
{
"salesID": "4404Bob Builder210032011-02-18T15:52:21+0.00",
"id": "4404",
"firstName": "Bob",
"lastName": "Builder",
"address": "181 Construction Road, Adelaide, NSW",
"postal": "21003",
"country": "New Zealand",
"creationDate": "2011-02-18T15:52:21+0.00",
"accountType": "personal",
"miles": 17469
}
]
A non-dynamic way to do this is like:
payload[0].country
I believe the best way to do this is with the filter function. The below option gives me the entire object, but I just want the country field.
payload filter ($.id == "4404")
Map function seems to be overkill for this since I only want the value, itself. I must be missing the syntax to get at the country field.
I did some more investigating, and this solution got me close enough to what I wanted:
https://stackoverflow.com/a/43488566/11995149
For my code example using filter, I had to surround the whole expression in parenthesis, and then I can access the field with a dot reference,
but below code gives String value as a single record within an array:
(payload filter ($.id == "4404")).country
[
"New Zealand"
]
In my case, I know that just one result will be returned from the filtered payload, so I could get just the String value with:
(payload filter ($.id == "4404"))[0].country
enter image description here
Can you try any of below options:
1) (payload groupBy ((item, index) -> item.id))["4404"][0].country
OR
2) (payload map ((item, index) -> if(item.id == "4404") item.country else ""))[0]
Thanks,
Ashish

Mule Dataweave : Filter in combination with Default function not working

Im using Mule dataweave, here is my request, i want to filter my request to code == "P" if request containing Code = p not present then always default to code == "C" because code = C always present in the incoming request.
Request:
{
"addresses": [
{
"contact": 0,
"code": "P",
"TypeDescription": "POSTAL",
"postcode": "1007",
"State": "TamilNadu",
"Description": "Test",
},
{
"contact": 1,
"code": "C",
"TypeDescription": "PHYSICAL",
"postcode": "Bangalore",
"State": "",
"Description": "NEW",
}
]
}
Dataweave:
%dw 1.0
%output application/json
---
payload.addresses filter $.code == "P"
It is working fine and filtering out the P containing list, but when try filter in combination with default it dont wont.
I have tried as below in dataweave payload.addresses filter $.code == "P1" default "anything here as of now".
Since in the above response i'm filtering P1 which is not present in the request hence im expecting the default stuff in the response. But it is not working.
Note: Using when and otherwise, bringing the list twice. I need only one list as response either Code P containing list or C containing list.
Using MUle 3.8.5 V. Please let me know your thoughts. Thanks in advance.
From what I understand, Dataweave works from right to left, so in this case is evaluating like this:
'"P1" default "anything here as of now"' resolves to "P1"
run 'filter $.code == "P1"' over 'payload.addresses'
Of course that first option will always evaluate to "P1" so the default is essentially ignored. You need to use parentheses around the rest of the expression, then follow it with default. Now it will evaluate the parentheses then the default expression.
eg: (payload.addresses filter $.code == "P1") default "anything here as of now"
This is not a good example though, as filter will return an empty list if none of the options match, not null, so the default won't work. You could try something like in this answer: https://stackoverflow.com/a/44431546/2614624
Create a default value in local variable and use an OR condition while filtering.
Example below.
{
biggerThanTwo: [0, 1, 2, 3, 4, 5] filter (($ > 2) OR ($ ==1))
}
Here is my Solution. As #Clinton mentioned Filter returns Empty array not null, here is the way it worked out
%dw 1.0
%output application/java
%var conditionalRoute = payload.addresses filter $.addressTypeCode == "P"
conditionalRoute when conditionalRoute != [] otherwise "anything here as of now"