Trouble parsing URL encoded URI using read(data, "application/x-www-form-urlencoded") function in Dataweave 2.0 - mule

I have this url from which I am trying to retrieve the queryParams.
%dw 2.0
output application/json
import * from dw::core::URL
---
{
queryParams : read(parseURI("https://google.com?token=abc%2FL℅2Bxyz%2F").query default "", "application/x-www-form-urlencoded")
}
Output
{
"queryParams": {
"token": "abc/L xyz/"
}
}
Based on url encoding, / is %2F and + is %2B. But in the output I got a space in place of +.
I understand that parseURI is basically decoding the URI to just "token=abc/L+xyz/", but why does applying a read(data, "application/x-www-form-urlencoded") on top of it change the value to abc/L xyz/?
I am trying understand this behavior as I am trying to retrieve the original URI. For that I did the below
encodeURIComponent(read(parseURI("https://google.com?token=abc%2FL℅2Bxyz%2F").query default "", "application/x-www-form-urlencoded").token)
The output for the above line of code is "abc%2F%20xyz%2F". So %2B is getting converted to %20.

Firstly there are some issues with your script as shown. It doesn't to work as is because the package import is case sensitive. Also it uses the character ℅ instead of the percentage character % for 2B. After fixing those issues what you described can be reproduced.
I changed the script to show both the output of parseURI() and read(). You can see that parseURI() returns a + which is expected. Then using read() with media type "application/x-www-form-urlencoded" you are telling read that the content is to be parsed in urlencoded form format. That implies converting a + into a space character to obtain the actual URL. See this answer about URL encoding for more details.
Using encodeURI() to reencode the output will return %20 instead of +, which seems to be more canonical.
If you absolutely need the original string you may want to use string operations or regular expressions to decompose the query string into the token value.
Example:
Script
%dw 2.0
output application/json
import * from dw::core::URL
var parsed=parseURI("https://google.com?token=abc%2FL%2Bxyz%2F").query
---
{
parsed : parsed,
read: read(parsed default "", "application/x-www-form-urlencoded").token,
encoded: encodeURI(read(parsed default "", "application/x-www-form-urlencoded").token)
}
Output
{
"parsed": "token=abc/L+xyz/",
"read": "abc/L xyz/",
"encoded": "abc/L%20xyz/"
}
Example of obtaining the token using string operations (splitBy()) and some mapping.
%dw 2.0
output application/json
import * from dw::core::URL
var parsed=parseURI("https://google.com?token=abc%2FL%2Bxyz%2F&other=1234").query
---
{
queryParam : parsed splitBy "&"
map ( (($ splitBy "=")[0]): ($ splitBy "=")[1] )
reduce ((item, acc={}) -> acc ++ item)
}
Output:
{
"queryParam": {
"token": "abc/L+xyz/",
"other": "1234"
}
}

Related

How do I output a plain text string to a variable on Anypoint Platform with Dataweave?

I feel like I'm losing my mind because I can't figure out how to do something as simple as iterate over an object, concatenate a plain text string, and output the result to a variable. Here is something similar I've done which works fine:
%dw 2.0
output application/xml
var promptParams = attributes.queryParams filterObject($$ startsWith "PROMPT") orderBy($$)
---
{
RESULT: {
Prompts:
promptParams mapObject(promptValue, promptName, index) -> {
PROMPT: {
UniquePromptName: promptName,
FieldValue: promptValue
}
}
}
}
So in this case I filter the url query string parameters to get only the ones I want, then I iterate over those and construct the xml output. The problem I'm having is that if I try to do the same thing but output a plain text string to a variable, I can't get anything to work.
Basically what I want is to go from this input:
https://example.com?PROMPT1=foo&PROMPT2=bar&PROMPT3=lorem&PROMPT4=ipsum&utm_source=Dolor&utm_campaign=SitAmet
to this output stored in a flow variable:
foo!bar!lorem!ipsum
I must be missing something basic because it can't be that hard to accomplish this. What am I doing wrong?
It should be something like this:
%dw 2.0
output text/plain
var promptParams = attributes.queryParams filterObject($$ startsWith "PROMPT")
---
promptParams pluck($) reduce ($$ ++ "!" ++ $)
Output:
foo!bar!lorem!ipsum
You asked for text plain, but I would recommend application/java if you are usign the variable inside the flow.
%dw 2.0
output text/plain
var promptParams = (((payload.message splitBy "?")[1]) splitBy "&") //stored url //in payload.message
---
promptParams map {
a: ($ splitBy "=")[1]
}.a joinBy "!"
you can use pluck and joinBy and just make sure that you have the target set to a variable if you are using the transform message component.
<ee:transform doc:name="Transform Message" >
<ee:variables >
<ee:set-variable variableName="promptAttributes" ><![CDATA[%dw 2.0
output text/plain
---
(attributes.queryParams[?($$ startsWith "PROMPT")] pluck $) joinBy "!"]] </ee:set-variable>
</ee:variables>
</ee:transform>

Dataweave code not working mule 4 which was working in mule3

The following code snippet was working in 3.8 but not working in 4.1:
output application/java
var csv = payload
---
(csv map $ reduce ((val,acc) -> ((acc) ++ ((val)) ))) map ($ replace ',' with '\t')
input payload:
{"D01":{"AK":"D,01,AK,0,0,0,0,0,-2.89,0.00,0,0,0,0,0",
"AL":"D,01,AL,829.23,18506.35,0,0.00,0,-6610.91,0.00,0,0,0,159.66,-1.94"},
"D02.1":{"AK":"D,02.1,AK,0,0,0,0,0,-6.76,0.00,0,0,0,0,0",
"AL":"D,02.1,AL,7733.77,304148.90,0,0.00,0,-42791.15,0.00,0,0,0,1347.09,-8.88"}
}
enter code here
Expected Output:
[
"D\t01\tAK\t0\t0\t0\t0\t0\t-2.89\t0.00\t0\t0\t0\t0\t0",
"D\t01\tAL\t829.23\t18506.35\t0\t0.00\t0\t-6610.91\t0.00\t0\t0\t0\t159.66\t-1.94",
"D\t02.1\tAK\t0\t0\t0\t0\t0\t-6.76\t0.00\t0\t0\t0\t0\t0",
"D\t02.1\tAL\t7733.77\t304148.90\t0\t0.00\t0\t-42791.15\t0.00\t0\t0\t0\t1347.09\t-8.88"
]
Am getting below Error:
org.mule.runtime.core.internal.exception.OnErrorPropagateHandler:
Message : "You called the function 'map' with these arguments:
1: String ("{\"D01\":{\"AK\":\"D,01,AK,0,0,0,0,0,-2.89,0.00,0,0,0,0,0\", \"AL\":\"D,01,A...)
2: Function (($:Any, $$:Any) -> ???)
But it expects arguments of these types:
1: Array
2: Function
13| (csv map $ reduce ((val,acc) -> ((acc) ++ ((val)) ))) map ($ replace ',' with '\t')
^^^^^^^^^
I think the problem there is, map in DataWeave 2 does not work on Object (See changes). DataWeave 1 allowed that and hence your code was valid for DataWeave 1.
Based on the output with DataWeave 1, I think you can use following code for DataWeave 2 -
%dw 2.0
output application/java
var csv = payload
---
flatten ((csv pluck $) map ($ pluck $)) map ($ replace ',' with '\t')
pluck will split an object into two arrays - values ($) and keys ($$).

How to send a byte array as a part of Json with karate framework

I have an endpoint who consumes Json with 2 attributes, like
{id='12344', data=byte_array}
so I've wrote a test
Feature: submitted request
Scenario: submitted request
* def convertToBytes =
"""
function(arg) {
var StreamUtils = Java.type('my.utils.StreamUtils');
// it reads stream and convert it to a byte array
return StreamUtils.getBytes(arg);
}
"""
Given url 'http://my-server/post'
And def image = convertToBytes(read('classpath:images/image_1.jpg'));
And request {id:1, data: "#(image)"}
When method POST
Then status 200
However is got an exception form karate without much details
ERROR com.intuit.karate - http request failed: [B cannot be cast to [Ljava.lang.Object;
Any hits how to submit byte arrays as a part of Json with karate?
I don't think you can do that. Either the whole request should be binary (byte-array) or you do a multi-part request, where binary is Base64 encoded. As far as I know you can't put binary inside JSON. There is something called Binary JSON though.
EDIT: after assuming that the byte[] has to be Base64 encoded:
Background:
* url demoBaseUrl
* def Base64 = Java.type('java.util.Base64')
Scenario: json with byte-array
Given path 'echo', 'binary'
And def encoded = Base64.encoder.encodeToString('hello'.bytes);
And request { message: 'hello', data: '#(encoded)' }
When method post
Then status 200
And def expected = Base64.encoder.encodeToString('world'.bytes);
And match response == { message: 'world', data: '#(expected)' }
I just added this test to the Karate demos, and it is working fine. Here is the commit.

POSTMAN: Extracting Values from body

I'm trying to recreate a scenario with the postman and there is a _csrf value in the previous GET request response body to be passed with the next POST request.
I Can't find a way to extract the value from POSTMAN.
NOTE: What I want is something similar to Regular Expression Extractor in Jmeter.If you have any Idea about extracting a value form the response body and setting it to a variable. Please let me know.
Cheers,
Muditha
This might help you https://media.readthedocs.org/pdf/postman-quick-reference-guide/latest/postman-quick-reference-guide.pdf
They use Cheerio
2.2.5 How to parse a HTML response to extract a specific value?
Presumed you want to get the _csrf hidden field value for assertions or later use from the response below:
To parse and retrive the value, we will use the cherrio JavaScript library:
responseHTML = cheerio(pm.response.text());
console.log(responseHTML.find('[name="_csrf"]').val());
Cheerio is designed for non-browser use and implements a subset of the jQuery functionality. Read more about it at
https://github.com/cheeriojs/cheerio
responseHTML = cheerio(pm.response.text());
var po= responseHTML.find('[name="_csrf"]').val();
console.log(po);
pm.environment.set("token", po);
/* You need to set the environment in Postman and capture the CSRF token in variable "here po" using a get request. Next in post request the environment variable token can be used */
Just made this JS in post man to parse Without a REGEx. Hope it will help people in the futur
Text to parse : Json : Extract data-id :
{
"code": "OK",
"response": {
"append": {
"html": {
"< .folders": "<a class=\"folder\" href=\"/foobarfoo\" data-id=\"ToExtract\"><div><i class=\"far fa-fw fa-folder\"></i></div><div class=\"folder-name\">blabla</div><div><div class=\"badge\">0</div></div></a>"
}
}
}
}
console.log(responseBody.response);
var jsonData = JSON.parse(responseBody);
var iStart = responseBody.indexOf("response\":")+10;
var scenarioId = responseBody.substr(iStart,10);
var iEnd = scenarioId.indexOf("}");
var scenarioId = scenarioId.substr(0,iEnd);
console.log("scenarioId:" + scenarioId + "iStart: "+ iStart + " scenarioId : " + scenarioId);
pm.environment.set("scenario", scenarioId);

DataWeaver Mule Document section 1.1.2 and 3.4 examples not working?

Going through the Dataweaver documentaion
Link:https://developer.mulesoft.com/docs/dataweave#_attribute_selector_expressions
Section 3.4 Key Present
Trying out the example provide below .
Input:
<users>
<name>Mariano</name>
<name>Luis</name>
<name>Mariano</name>
</users>
Transform:
%dw 1.0
%input payload application/xml
%output application/xml
---
users: payload.users.name[?($ == "Mariano")]
when I try to give this expression in my DataWeaver it gives warning like cannot coerce a:string to a: array:(7,92).
Have given the same way mentioned in the document. Could anyone please advice.
Expected Response:
<?xml version="1.0" encoding="UTF-8"?>
<users>
<name>Mariano</name>
<name>Mariano</name>
</users>
Also in the document 1.1.2 string manipulation example wasn't working for me
%dw 1.0
%input payload application/xml
%output application/json
%function words(name) name splitBy " "
---
contacts: payload.users.*user map using (parts = words($.name)){
firstName: parts[0],
(secondName: parts[1]) when (sizeOf parts) > 2,
lastName: parts[-1],
email: "$((lower $.name) replace " " with ".")#acme.com.ar",
address: $.street
}
showing error like multiple marker at this line missing '}' no viable alternative at input email
Started learning and working out the examples provided. Thanks.
The example in the docs has a typo, there is an * missing before name (it should be fixed):
%dw 1.0
%input payload application/xml
%output application/xml
---
users: payload.users.*name[?($ == "Mariano")]
The problem is that XML doesn't have a built-in list representation, so the list is represented by multiple occurences of a tag. The expression *name returns a list with the occurrences of name, the expression [?($ == "Mariano")] it's like a filter (I prefer filter since it is easier to understand).
The cryptic error message appears because the operator applies to a list, but payload.users.name returns the first appeareance of name. (That's why it says cannot coerce string to array).