Mule DataWeave JSON content is null - mule

Question

Mule version: 3.7.3
Anypoint Studio: 5.4.0
To learn DataWeave, I've created a simple flow that executes twitter search, loops through each tweet, attempts to use DataWeave to condense the payload, then logs the results. The issue I'm running into is that the json seems to be null coming out of DW.
Tweet Text before DW:
Welcome to the world of #APILedConnectivity Lebron James! Your request has been processed by #MuleSoft #AnypointPlatform # 1986
The payload coming out of DW is not null:
org.mule.api.processor.LoggerMessageProcessor: DW Payload: com.mulesoft.weave.mule.WeaveMessageProcessor$WeaveOutputHandler#1657d772
However, the 'tweet' json element is null when outputting to logs:
org.mule.api.processor.LoggerMessageProcessor: Found Tweet: null
Here's my sample mule flow:
<?xml version="1.0" encoding="UTF-8"?>
<mule xmlns:dw="http://www.mulesoft.org/schema/mule/ee/dw" xmlns:metadata="http://www.mulesoft.org/schema/mule/metadata" xmlns:json="http://www.mulesoft.org/schema/mule/json" xmlns:http="http://www.mulesoft.org/schema/mule/http" xmlns:twitter="http://www.mulesoft.org/schema/mule/twitter" xmlns="http://www.mulesoft.org/schema/mule/core" xmlns:doc="http://www.mulesoft.org/schema/mule/documentation"
xmlns:spring="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-current.xsd
http://www.mulesoft.org/schema/mule/core http://www.mulesoft.org/schema/mule/core/current/mule.xsd
http://www.mulesoft.org/schema/mule/http http://www.mulesoft.org/schema/mule/http/current/mule-http.xsd
http://www.mulesoft.org/schema/mule/twitter http://www.mulesoft.org/schema/mule/twitter/current/mule-twitter.xsd
http://www.mulesoft.org/schema/mule/ee/dw http://www.mulesoft.org/schema/mule/ee/dw/current/dw.xsd
http://www.mulesoft.org/schema/mule/json http://www.mulesoft.org/schema/mule/json/current/mule-json.xsd">
<flow name="test-dataweaveFlow">
<http:listener config-ref="HTTP_Listener_Configuration" path="/test" doc:name="HTTP"/>
<twitter:search config-ref="TwitterConnection" query="#[json:hashtag]" doc:name="Twitter"/>
<foreach collection="#[payload.tweets]" doc:name="For Each">
<logger message="Payload before DW: #[payload.text]" level="INFO" doc:name="Logger"/>
<dw:transform-message metadata:id="99e88418-ce6a-4d18-a900-6c35d7f36c83" doc:name="Transform Message">
<dw:input-payload doc:sample="QueryResult.dwl"/>
<dw:set-payload><![CDATA[%dw 1.0
%output application/json
---
{
tweet: {
screenName: payload.user.screenName,
tweetId: payload.id,
tweetContent: payload.text
}
}
]]></dw:set-payload>
</dw:transform-message>
<logger message="DW Payload: #[payload]" level="INFO" doc:name="Logger"/>
<logger message="Found Tweet: #[payload.tweet]" level="INFO" doc:name="Logger"/>
</foreach>
<json:object-to-json-transformer doc:name="Object to JSON"/>
</flow>
</mule>
Screenshot of the mapping:
Any ideas where I might be messing things up?

Answer 1

Don't put the DataWeave transformer in the for each scope!
You can iterate in DataWeave, that will give you a better result probably.
You will create a map from the payload, the map operator accepts a collection.
The twitter connector will give you a collection of JSON objects.
The clue here is, drag the DataWeave transformer after the Twitter connector.
The input data in DataWeave will be provided and you will have autocomplete in you DataWeave script.
Docs: https://docs.mulesoft.com/mule-user-guide/v/3.7/dataweave-reference-documentation
Some examples in how to map form JSON here:
https://docs.mulesoft.com/mule-user-guide/v/3.7/dataweave-examples#json-to-xml