mulesoft dataweave 2.0 mapObject recursion match JSON values - mule

I have a kafka message listener connector and after that there is the connector 'Avro Scheme' by mulesoft. Now this Avro Scheme connector replaces parts of my JSON where there is a validation error. I'm trying to see where & what exception occured.
I have following payload:
[
{
"movementId": "a4fa404e-6983-11ed-a1eb-0242ac120002",
"movementNumber": "123456789",
"movementDate": "2022-11-22",
"originLocationId": {
"localizedMessage": "Invalid UUID string: ipsum et",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string: ipsum et",
"stackTrace": [
],
"suppressed": [
]
},
"originLocationCode": "jos",
"actDepartureDateTime": "2022-11-21T11:08:17.057",
"estArrivalDateTime": "2022-11-21T11:08:17.057",
"delays": null,
"event": {
"id": {
"localizedMessage": "Invalid UUID string: Excepteur",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string: Excepteur",
"stackTrace": [
],
"suppressed": [
]
},
"created": "2022-11-21T11:08:17.057",
"source": {
"application": "TEST",
"server": "aute reprehenderit et",
"instance": "exercitation"
}
}
}
]
Now I need to fetch the paths of all the JSON objects where the value 'AvroReadingException' occurs. It doesn't really matter if it is the full path to the JSON key, but something to identify where the exception was thrown.
In given example the expected result could be:
[
{
"originLocationId": "Invalid UUID string: ipsum et"
},
{
"event.id": "Invalid UUID string: Excepteur"
}
[
I started with following test, but cannot seem to get it to work:
%dw 2.0
output application/json
fun test(root, value, opt) = do {
value match {
case arr is Array -> value map (value, index) -> test(null, value, null)
case obj is Object -> value mapObject { ($$): test($$, $, root) }
else ->
if (value is String and (value contains 'AvroReadingException'))
value ++ "------" ++ opt
else
"++++"
}
}
---
test(null, payload, null)
Thanks in advance!

You can use the following. I have broken the logic in multiple functions to make it more readable.
%dw 2.0
output application/json
// This is just a small utility function to easily concatenate strings to get path
fun appendToPath(currentPath, pathToAppend) =
if(isEmpty(currentPath)) pathToAppend
else currentPath ++ "." ++ pathToAppend
fun isRequiredExceptionObject(data) =
(data is Object) and (data.kind == "AvroReadingException")
/**
* This is the recursive function. The path is basically the path
* that has been collected till now. For example. In your case
* for calculating "event.id", when the function is called recursively this value will be "event"
*/
fun getAvroReadingExceptions(validationPayload: Object, path="") =
entriesOf(validationPayload)
reduce ((item, accumulator = {}) ->
item.value match {
case value if(isRequiredExceptionObject(value)) -> {
(accumulator),
(path appendToPath item.key): value.message
}
case is Object -> {
(accumulator),
(getAvroReadingExceptions(item.value, path appendToPath item.key))
}
case is Array -> {
(accumulator),
(item.value map getAvroReadingExceptions($, path appendToPath item.key ++ "[$($$)]"))
}
else -> accumulator
}
)
---
payload map getAvroReadingExceptions($)
This also handles internal arrays. I have used the following input and added an array arrayTest which has 3 elements, among which 2 has a field that is AvroReadingException.
[
{
"movementId": "a4fa404e-6983-11ed-a1eb-0242ac120002",
"movementNumber": "123456789",
"movementDate": "2022-11-22",
"originLocationId": {
"localizedMessage": "Invalid UUID string: ipsum et",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string: ipsum et",
"stackTrace": [
],
"suppressed": [
]
},
"originLocationCode": "jos",
"actDepartureDateTime": "2022-11-21T11:08:17.057",
"estArrivalDateTime": "2022-11-21T11:08:17.057",
"delays": null,
"event": {
"id": {
"localizedMessage": "Invalid UUID string: Excepteur",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string: Excepteur",
"stackTrace": [
],
"suppressed": [
]
},
"created": "2022-11-21T11:08:17.057",
"source": {
"application": "TEST",
"server": "aute reprehenderit et",
"instance": "exercitation"
}
},
"arrayTest": [
{
"field1": "sdf",
"field2": {
"localizedMessage": "Invalid UUID string in array: element3",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string in array: element1",
"stackTrace": [
],
"suppressed": [
]
}
},
{
"field1": "sdf",
"field2": "sdf"
},
{
"field1": "sdf",
"field2": {
"localizedMessage": "Invalid UUID string in array: element3",
"kind": "AvroReadingException",
"cause": null,
"message": "Invalid UUID string in array: element3",
"stackTrace": [
],
"suppressed": [
]
}
}
]
}
]
This is the output from the Dataweave
[
{
"originLocationId": "Invalid UUID string: ipsum et",
"event.id": "Invalid UUID string: Excepteur",
"arrayTest[0].field2": "Invalid UUID string in array: element1",
"arrayTest[2].field2": "Invalid UUID string in array: element3"
}
]

Related

How to match string and ignore the case in karate?

There is a case where one value sometimes is lower case and sometimes it's upper case. this is the response coming from an API and we have to match if every field in response is correct by ignoring some values. The error text in response sometimes has one keyword in lower and some scenarios it is upper case. How we can ignore one keyword in a string to not match? I don't want to ignore whole text as it works fine if I ignore whole string but is it possible to ignore one keyword only?
Scenario: string matching
* def test =
"""
{
"sourceType": "Error",
"id": "123456",
"type": "searchuser",
"total": 0,
"value": [
{
"details": "this is the user search case",
"source": {
"sourceType": "Error",
"id": "77200203043",
"issue": [
{
"severity": "high",
"code": "678",
"message": {
"text": "No matching User details found"
},
"errorCode": "ERROR401"
}
]
},
"user": {
"status": "active"
}
}
]
}
"""
* match test ==
"""
{
"sourceType": "Error",
"id": "#present",
"type": "searchuser",
"total": 0,
"value": [
{
"details": "#present",
"source": {
"sourceType": "Error",
"id": "#ignore",
"issue": [
{
"severity": "high",
"code": "678",
"message": {
"text": "No matching User details found"
},
"errorCode": "ERROR401"
}
]
},
"user": {
"status": "active"
}
}
]
}
"""
How to ignore the case only for user here? I tried below but it treats #ignore as a value.
"text": "No matching #ignore details found"
I'm not looking at your payload dump but providing a simple example. Use karate.lowerCase():
* def response = { foo: 'Bar' }
* match karate.lowerCase(response) == { foo: 'bar' }
EDIT: you can also extract one value at a time and do a check only for that:
* def response = { foo: 'Bar' }
* def foo = response.foo
* match karate.lowerCase(foo) == 'bar'

how to remove objects that have all keys with null values in dataweave?

I have this below payload and I want to remove object where all the keys have ALL empty values,
[
{
"Order" : "123",
"Product" : "456"
},
{
"Order" : "",
"Product" : ""
}
]
This is what the output should be like,
[
{
"Order" : "123",
"Product" : "456"
}
]
None of the posted solutions handle things like nested structures or arrays, so I thought I'd throw this recursive solution in the ring. This allows us to traverse the entire structure of the object until we hit the first non-null field.
%dw 2.0
output application/json
import everyEntry from dw::core::Objects
import every from dw::core::Arrays
var allFieldsNull = (obj: Any) ->
obj match {
case is Object -> obj everyEntry (allFieldsNull($))
case is Array -> (sizeOf(obj) == 0) or (obj every allFieldsNull($))
//case is Array -> false
else -> isEmpty(obj)
}
---
payload filter !allFieldsNull($)
If you wanted to consider an empty array as enough to keep the object since that technically isn't null, you would just need to comment out the case is Array line and uncomment the one below it.
Input:
[
{
"Order" : "123",
"Product" : "456"
},
{
"Order" : "",
"Product" : "",
"Address": {
"Field1": ""
},
"Test": [
{
"Order" : "",
"Product" : "",
"Address": {
"Field1": ""
}
}
]
},
{
"Order" : null,
"Product" : null,
"Address": {
"Field1": null
},
"Test": [
{
"Order" : null,
"Product" : null,
"Address": {
"Field1": "A value even in a deeply nested field means I show up"
}
}
]
}
]
output:
[
{
"Order": "123",
"Product": "456"
},
{
"Order": null,
"Product": null,
"Address": {
"Field1": null
},
"Test": [
{
"Order": null,
"Product": null,
"Address": {
"Field1": "A value even in a deeply nested field means I show up"
}
}
]
}
]
Would something like this work for you?
Input
[
{
"Order" : "123",
"Product" : "456"
},
{
"Order" : null,
"Product" : null
}
]
Script
%dw 2.0
output application/json
import * from dw::core::Objects
var valuesOfInputObjects = payload map { ($ takeWhile((value, key) -> value == null))}
---
payload -- valuesOfInputObjects
output
[
{
"Order": "123",
"Product": "456"
}
]
You can filter by a condition, using the everyEntry() function to see that not all values are empty.
%dw 2.0
output application/json
import * from dw::core::Objects
---
payload filter ($ someEntry (value, key) -> !isEmpty(value))

Karate - Nested JSON object schema validation causes KarateException

Feature: Test Karate schema validation
Scenario: Test nested json objects
* def response = read('tasks.json')
* def schema = { ab: "##[] string", c: "##[] string" }
* match response ==
"""
{
id: '#string',
name: '#string',
obj1: '#(schema)' ,
obj2: '##(schema)' ,
obj3: '#(schema)' ,
obj4: '#null'
}
"""
Following is json file used (tasks.json)
{
"id": "ad:p2:53456:4634:yu",
"name": "name",
"obj1": {
"ab": [
"test"
],
"c": null
},
"obj2": null,
"obj3": {
"ab": [
"tester"
],
"c": [
"t1", "t2"
]
},
"obj4": null
}
Error: com.intuit.karate.exception.KarateException: javascript evaluation failed: string, ReferenceError: "string" is not defined in at line number 1
I have tried multiple ways like :
obj1: '#(^schema)',
obj1: '#object schema'
but not able to fix the issue.
It should be ##[] #string , read the docs: https://github.com/intuit/karate#schema-validation

Transformation of JSON data in mule 4

I have a requirement wherein I have to convert JSON data from one format to other.
I have to fetch corresponding values of JSON array and make them a key value pair.
Below are the required details:
Input:
"Headers": {
"Header": [
{
"Key": "SellerOrganization",
"Value": "XYZ"
},
{
"Key": "SellerType",
"Value": "B2C"
},
{
"Key": "Region",
"Value": "SOUTH"
},
{
"Key": "OrderType",
"Value": "RETURN"
},
{
"Key": "InvoiceType",
"Value": ""
},
{
"Key": "EventType",
"Value": "Created"
},
{
"Key": "EntryType",
"Value": "Call Center"
}
]
}
Expected Output:
{
SellerOrganization:XYZ,
SellerType: B2C,
Region:SOUTH,
OrderType:RETURN,
InvoiceType:"",
EventType:Created,
EntryType:Call Center
}
You can use the dynamic object that it will basically do what you want.
%dw 2.0
output application/json
---
{
(payload.Headers.Header map ((item, index) -> {
(item.Key): item.Value
})
)
}
You can take advantage of reduce function here which will let you convert your array to an key, value pair object
%dw 2.0
output application/json
---
payload.Header reduce ((item, acc = {}) -> acc ++ {
(item.Key): item.Value
})

MuleSoft transform gives error while transforming using DWL for JSON payload

Im am new to MuleSoft.I am trying to transform a JSON payload,using transform.
I want to transform my payload as below
Input:
{
"ResponseStatus": {
"Status": "SUCCESS",
"StatusText": "SUCCESS"
},
"Processes": {
"Process": [
{
"ProcessId": "1234567",
"ProcessProperties": {
"Property": [
{
"Name": "XXXXXXXXXXX",
"Value": "11111111",
"Desc": "YYYYYYYY"
},
{
"Name": "AAAAAAAAA",
"Value": "2222222",
"Desc": "BBBBBBBB"
},
{
"Name": "QQQQQQQQQ",
"Value": "#######",
"Desc": "CCCCCCCC"
},
{
"Name": "NNNNNNN",
"Value": "IIIIIIII",
"Desc": "UYUYUYUY"
}
]
},
"EditMode": "CCCCCC",
"ProcessType": "ABCD",
"AppName": "VFVFVGBG",
"StatusHistory": {
"STS": [
{
"Sts": "COMPLETED"
}
]
}
}
]
}
}
Output:
[
{
"ProcessId": "1234567",
"AAAAAAAAA": "2222222",
"QQQQQQQQQ": "#######"
}
]
I have read DWL reference from below Mulesoft link.Also reffered this SO link.
Below is what I have tried so far,
%dw 1.0
%output application/json
---
{
"ProcessId": (payload.Processes.Process.ProcessId)[0],
AAAAAAAAA: {
(payload.Processes.Process.ProcessProperties.Property mapObject {
($.Name):$.Value when $.Name =="AAAAAAAAA" otherwise ""
})
},
QQQQQQQQQ: {
(payload.Processes.Process.ProcessProperties.Property mapObject {
($.Name):$.Value when $.Name =="QQQQQQQQQ" otherwise ""
})
}
}
I am still not able to get the desired output.
It gives me "Cannot coerce a :array to a :key"
Can anyone please help me?
The "property" json element in your input json is "Array",which it is not able to parse to a single value.
Please try below snippet and let me know if that gives your deisred o/p.
payload.Processes.Process map (
(val , index) ->
{"ProcessId":(payload.Processes.Process.ProcessId)[0]
,
(val.ProcessProperties.Property map {
(($.Name) : $.Value) when $.Name =='AAAAAAAAA' }
),
(val.ProcessProperties.Property map {
(($.Name) : $.Value) when $.Name =='QQQQQQQQQ' }
)
}
)