The schema shows that a field is a "system.string" and it has a variable number of elements, like a list of lists. What I would like to do is convert this into a normal string and store that in a field. Just tostring() does not cut it. It is like I have to unpack and then concatenate. How can I do this? You could reference each element with Tags[0], Tags[1].... but there is a variable number of indexes.
example: How can one expland MoreData into a flat string?
datatable(Date:datetime, Event:string, MoreData:dynamic) [
datetime(1910-06-11), "Born", dynamic(["value5", "value6"]),
datetime(1930-01-01), "Enters Ecole Navale", dynamic(["value5", "value6"]),
datetime(1953-01-01), "Published first book", dynamic(["value5", "value6"]),
datetime(1997-06-25), "Died", dynamic(["value5", "value6"]),
]
Suppose you wanted to generate a result of '"value5", "value6"' as a string?
datatable(Date:datetime, Event:string, MoreData:dynamic) [
datetime(1910-06-11), "Born", dynamic(["value5", "value6"]),
datetime(1930-01-01), "Enters Ecole Navale", dynamic(["value5", "value6"]),
datetime(1953-01-01), "Published first book", dynamic(["value5", "value6"]),
datetime(1997-06-25), "Died", dynamic(["value5", "value6"]),
]
| extend result = array_strcat(MoreData,',')
Date
Event
MoreData
result
1910-06-11T00:00:00Z
Born
["value5","value6"]
value5,value6
1930-01-01T00:00:00Z
Enters Ecole Navale
["value5","value6"]
value5,value6
1953-01-01T00:00:00Z
Published first book
["value5","value6"]
value5,value6
1997-06-25T00:00:00Z
Died
["value5","value6"]
value5,value6
Fiddle
Related
I have a data weave transformation converting an excel file to JSON, I had to change the value of an element (column of the file) as per the key-value pair values stored in a variable.
Please let me know how to achieve this.
Below is my data weave which converts 600 rows in the file to a JSON. However, I need to change the value for Brand as per the key-value pair mapping I stored in a variable.
%dw 2.0
output application/json
---
payload map(payload01,index01)->{
city: payload01.City,
province: payload01.Province,
phone: payload01.Phone,
fax: payload01.FAX,
email: payload01.EMAIL,
Brand: payload01.'Fuel Brand'
}
I understand that you want to use the value in attribute 'Fuel Brand' of the input payload to be used as the index in a variable:
%dw 2.0
output application/json
---
payload."Sheet Name" map(payload01,index01)-> {
city: payload01.City,
...
Brand: vars.brandsMapping[payload01.'Fuel Brand']
}
For example if the input is (note: I removed other attributes to simplify the example):
[{City=City A, Fuel Brand=brand1}, {City=City B, Fuel Brand=brand3}]
And the variable vars.brandsMapping contains:
{brand1=The brand1, brand2=The brand2, brand3=The brand3}
The output will be:
[
{
"city": "City A",
"Brand": "The brand1"
},
{
"city": "City B",
"Brand": "The brand3"
}
]
UPDATE: Since you clarified that you want to a dynamic mapping, the method that can be used for that is mentioned at the documentation page: https://docs.mulesoft.com/mule-runtime/4.1/dataweave-cookbook-map-based-on-an-external-definition
For the Mulesoft 4.2 Set Variable component, I want to assign a simple String value pulled from a single specific record from an incoming JSON payload.
In my case, taking just the value from the 'country' field from below example:
[
{
"salesID": "4404Bob Builder210032011-02-18T15:52:21+0.00",
"id": "4404",
"firstName": "Bob",
"lastName": "Builder",
"address": "181 Construction Road, Adelaide, NSW",
"postal": "21003",
"country": "New Zealand",
"creationDate": "2011-02-18T15:52:21+0.00",
"accountType": "personal",
"miles": 17469
}
]
A non-dynamic way to do this is like:
payload[0].country
I believe the best way to do this is with the filter function. The below option gives me the entire object, but I just want the country field.
payload filter ($.id == "4404")
Map function seems to be overkill for this since I only want the value, itself. I must be missing the syntax to get at the country field.
I did some more investigating, and this solution got me close enough to what I wanted:
https://stackoverflow.com/a/43488566/11995149
For my code example using filter, I had to surround the whole expression in parenthesis, and then I can access the field with a dot reference,
but below code gives String value as a single record within an array:
(payload filter ($.id == "4404")).country
[
"New Zealand"
]
In my case, I know that just one result will be returned from the filtered payload, so I could get just the String value with:
(payload filter ($.id == "4404"))[0].country
enter image description here
Can you try any of below options:
1) (payload groupBy ((item, index) -> item.id))["4404"][0].country
OR
2) (payload map ((item, index) -> if(item.id == "4404") item.country else ""))[0]
Thanks,
Ashish
I have a string array in properties file, and I want to read its value in dataweave in JSON format.
The array in properties file is-
Countries = ["USA","England","Australia"]
in dataweave, I am using this-
%output application/json
---
{
countries: p('Countries')
}
Output I am getting is-
"countries": "[\"USA\",\"England\",\"Australia\"]",
Output I want is-
"countries": [
"USA",
"England",
"Australia"
]
I have tried with replace but no luck.
I also tried countries map $ as String after changing country array to Countries = ['USA','England','Australia'] but it says Invalid input 'S', expected :type or enclosedExpr
How to achieve this?
The problem is that properties file values are strings and not arrays so your expression is not interpreted. But don't worry you can use the read function
read(p('Countries'), "application/json"))
Need help constructing this mongo query.
So far I can query on the first level, but unable to do so at the next embedded level ("labels" > 2")
For example, the document structure looks like this:
> db.versions_20170420.findOne();
{
"_id" : ObjectId("54bf146b77ac503bbf0f0130"),
"account" : "foo",
"labels" : {
"1" : {
"name" : "one",
"color" : "color1"
},
"2" : {
"name" : "two",
"color" : "color2"
},
"3" : {
"name" : "three",
"color" : "color3"
}
},
"profile" : "bar",
"version" : NumberLong("201412192106")
This query I can filter at the first level (account, profile).
db.profile_versions_20170420.find({"account":"foo", "profile": "bar"}).pretty()
However, given this structure, I'm looking for documents where the "label" > "2". It doesn't look like "2" is a number, but a string. Is there a way to construct the mongo query to do that? Do I need to do some conversion?
If I correctly understand you and your data structure, "label" > "2" means that object labels must contain property labels.3, and it is easy to check with next code:
db.profile_versions_20170420.find(
{"account": "foo", "profile": "bar", "labels.3": {$exists: true}}
).pretty();
But it doesn't mean that your object contains at least 3 properties, because it is not $size function which calculates count of elements in array, and we cannot use $size because labels is object not array. Hence in our case, we only know that labels have property 3 even it is the only one property which labels contains.
You can improve find criteria:
db.profile_versions_20170420.find({
"account": "foo",
"profile": "bar",
"labels.1": {$exists: true},
"labels.2": {$exists: true},
"labels.3": {$exists: true}
}).pretty();
and ensure that labes contains elements 1, 2, 3, but in this case, you have to care about object structure on application level during insert/update/delete data in document.
As another option, you can update your db and add extra field labelsConut and after that you will be able to run query like this:
db.profile_versions_20170420.find(
{"account": "foo", "profile": "bar", "labelsConut": {$gt: 2}}
).pretty();
btw, it will work faster...
When I pass an input field of repeated record type into Bigquery UDF, it keeps saying that the input field is not found.
This is my 2 rows of data:
{"name":"cynthia", "Persons":[ { "name":"john","age":1},{"name":"jane","age":2} ]}
{"name":"jim","Persons":[ { "name":"mary","age":1},{"name":"joe","age":2} ]}
This is the schema of the data:
[
{"name":"name","type":"string"},
{"name":"Persons","mode":"repeated","type":"RECORD",
"fields":
[
{"name": "name","type": "STRING"},
{"name": "age","type": "INTEGER"}
]
}
]
And this is the query:
SELECT
name,maxts
FROM
js
(
//input table
[dw_test.clokTest_bag],
//input columns
name, Persons,
//output schema
"[
{name: 'name', type:'string'},
{name: 'maxts', type:'string'}
]",
//function
"function(r, emit)
{
emit({name: r.name, maxts: '2'});
}"
)
LIMIT 10
Error I got when trying to run the query:
Error: 5.3 - 15.6: Undefined input field Persons
Job ID: ord2-us-dc:job_IPGQQEOo6NHGUsoVvhqLZ8pVLMQ
Would someone please help?
Thank you.
In your list of input columns, list the leaf fields directly:
//input columns
name, Persons.name, Persons.age,
They'll still appear in their proper structure when you get the records in your UDF.