I have a document in which I have to check if the document has a heading called "Name". How can I check this using dataweave?
Example of document:
<h2 id="name">Name</h2>
<p>This Anypoint Template should serve as a foundation for setting an online sync of accounts from a Salesforce instance to many destination systems, using the Publish-subscribe pattern. Every time there is a new account or a change in an already existing one, the integration will poll for changes in the Salesforce source Org, publish the changes to a JMS topic and each subscriber will be responsible for updating the accounts in the target systems.</p>
DataWeave doesn't support HTML as a format however it does support XML. If the HTML input has a root tag you can parse as XML and use a recursive function to find if it has a key name and value matching.
%dw 2.0
output application/java
import every from dw::core::Arrays
import someEntry from dw::core::Objects
fun findString(x, keyName, s)= x match {
case o is Object -> o mapObject (($$): if ($$ as String == keyName and $ == s) true else findString($, keyName, s) ) someEntry (value, key) -> value == true
case s1 is String -> s1
case is Number -> false
case is Boolean -> false
case a is Array -> !((a map findString($, s)) every ($ == false)) // should not happen in XML
case is Null -> false
else -> false
}
---
findString(payload, "h2", "Name")
Input:
<html>
<body>
<h2 id="name">Name</h2>
<p>This Anypoint Template should serve as a foundation for setting an online sync of accounts from a Salesforce instance to many destination systems, using the Publish-subscribe pattern. Every time there is a new account or a change in an already existing one, the integration will poll for changes in the Salesforce source Org, publish the changes to a JMS topic and each subscriber will be responsible for updating the accounts in the target systems.</p>
</body>
</html>
Output: true
You can use xpath-extract from xml-module in Mule 4. You can use it to evaluate any XPath against your payload. For example the below XPath will work for you.
//*[matches(name(), '^h[1-6]$') and (text() = 'Name')]
It will match all tags with name that matches regex ^h[1-6]$, so h1 to h6 and has the value ( text() ) as Name. If you only want to look for h2 you can update regex part accordingly
<xml-module:xpath-extract
doc:name="Xpath extract"
doc:id="4b04a662-98cb-4dac-b4b5-18f8a9c6604e"
xpath="//*[matches(name(), '^h[1-6]$') and (text() = 'Name')]">
</xml-module:xpath-extract>
The module will return an array of string containing the results of the xpath. You can check if the output of this isEmpty().
Related
I'm trying to write lua plugin to extract and place some metadata from HTML page. It's a plugin for soupault static site generator and it requires lua version 2.5 for work. So no closures and no for loops in particular.
It is no sense to load you how this generator works because plugin exists as a small standalone file .lua and plugs in during the running of generator.
What is significant are the methods used by the plugin. These are used at the input and output of the plugin.
And more importantly, it is how to form and process the list of extracted tags with lua language.
Input data in the body of html page:
<site-meta-data>
#+title: post 1 title
#+subtitle: Post 1 subtitle
#+description: Post 1 decription
#+author: Billy
#+date: 2021-11-03
#+datepublished: 2021-06-02
#+usertags: inventory,errand
#+summary: Post 1 summary
#+id: 1-test1com
</site-meta-data>
And these are steps that plugin should take:
get strings between <site-meta-data></site-meta-data> tags into the
list get string from the list & split by first colon eg.
string.match(destination_number, "(.-):"))
create variable name as first word before colon and without #+ (eg. title)
create conditions for the next operation on this variable. If variable name which is
created from the string = current name, eg. meta_tag = title, insert
metatag with value or insert new tag with extracted value after
parent tag. I'll write by myself which tags should be inserted.
after processing of the list remove all between <site-meta-data></site-meta-data> including
itself.
As being near zero in lua scripting I just wrote the scratch of the script.
all_meta_tags = HTML.select_one(page, "site-meta-data")
all_meta_tags = HTML.parse(page, "site-meta-data")
print(all_meta_tags)
local index = 1
while all_meta_tags[index] do
meta_tag_line = all_meta_tags[index]
meta_tag = string.match(meta_tag_line, "(.-):")
meta_tag_content = string.match(meta_tag_line, ":(.*)")
meta_tag_content = strlower(String.trim(meta_tag_content))
meta_tag = Regex.replace(meta_tag, "#+", "")
if (meta_tag == "title") then
HTML.append_child(page, HTML.create_string('<meta name="title" content="value..">'))
elseif (meta_tag == 'subtitle') then
HTML.append_child(page, HTML.create_string('...'))
elseif (meta_tag == 'description') then
HTML.append_child(page, HTML.create_string('<meta name="description" content="meta_tag_value">'))
elseif (meta_tag == 'author') then
HTML.append_child(page, HTML.create_string('author...'))
elseif (meta_tag == 'date') then
HTML.append_child(page, HTML.create_string('<meta name="date" content="meta_tag_content">'))
end
index = index + 1
end
HTML.delete(HTML.select_one(page, "site-meta-data"))
Please, help me to modify the script to accomplish the above task. I'll correct what tags it should place.
Since version 4.0 soupault supports a pre-parse hook. So it's now possible to reimplement various types of front matter with that hook. The plugin should always put the rendered HTML before the page body. Plugin can be written as this example.
[hooks.pre-parse]
file = "hooks/org-mode-metadata.lua"
template = """
<h1 id="post-title">{{title}}</h1>
...
I'm new to migrating the mule 3 apps to mule 4 I have done almost conversion but one expression stopped my flow and not able to achieve the logic for it if anyone has an idea regarding the expression to transform please help me
Expression:
if(flowVars.maindata.keySet().contains(payload.idCaseNumber))
{
flowVars.temporary=[];
flowVars.maindata.get(payload.idCaseNumber).add(map);
}
else
{
flowVars.temporary.add(previousdata);
vars.maindata.put(payload.idCaseNumber,temporary);
}
I have tried up to my knowledge on the above code but still I'm getting problem
flowVars.maindata.get(payload.idCaseNumber).add(map);
In Mule 3 the expression language is MEL. In Mule 4 it is DataWeave 2.0. You can't just translate directly. MEL is an imperative scripting language, similar to a subset of Java and it is easy to call Java methods. DataWeave 2.0 is a functional language. Furthermore Mule 4 operations (example: a , , etc) can only return one value, which can be assigned to the payload or to one variable.
For your snippet I'll assume that maindata is a map. You can use two set-variable to assign each variable:
<set-variable variableName="temporary" value="#[ if( namesOf(vars.maindata) contains payload.idCaseNumber ) [] else vars.temporary ++ **previousdata** ]" />
I don't know exactly what do you use for previousdata.
To update the variable maindata it is probably a good match for the update operator, in a separate or Transform operation, with the same condition than for vars.temporary.
Update:
I'll assume vars.maindata is a map, which DataWeave will consider an object, and each element is a list. As an example of doing an 'upsert' operation with a dynamic selector:
%dw 2.0
output application/java
var temporary=[5]
var maindata={ a:[1,2,3,4] }
var myKey="a"
---
maindata update {
case data at ."$(myKey)"! -> if (data != null) data ++ temporary else temporary
}
You could replace in above script the DataWeave var temporary with the expression from my example above, and the other DataWeave variables with the Mule variables (vars.name) or payload. If you change in above example myKey to have value "b" you will see that key being added.
So here is what I want as a module in Pseudo Code:
IF UseCustom, Create AWS Launch Config With One Custom EBS Device and One Generic EBS Device
ELSE Create AWS Launch Config With One Generic EBS Device
I am aware that I can use the 'count' function within a resource to decide whether it is created or not... So I currently have:
resource aws_launch_configuration "basic_launch_config" {
count = var.boolean ? 0 : 1
blah
}
resource aws_launch_configuration "custom_launch_config" {
count = var.boolean ? 1 : 0
blah
blah
}
Which is great, now it creates the right Launch configuration based on my 'boolean' variable... But in order to then create the AutoScalingGroup using that Launch Configuration, I need the Launch Configuration Name. I know what you're thinking, just output it and grab it, you moron! Well of course I'm outputting it:
output "name" {
description = "The Name of the Default Launch Configuration"
value = aws_launch_configuration.basic_launch_config.*.name
}
output "name" {
description = "The Name of the Custom Launch Configuration"
value = aws_launch_configuration.custom_launch_config.*.name
}
But how the heck do I know from the higher area that I'm calling the module that creates the Launch Configuration and Then the Auto Scaling Group which output to use for passing into the ASG???
Is there a different way to grab the value I want that I'm overlooking? I'm new to Terraform and the whole no real conditional thing is really throwing me for a loop.
Terraform: How to conditionally assign an EBS volume to an ECS Cluster
This seemed to be the cleanest way I could find, using a ternary operator:
output "name {
description = "The Name of the Launch Configuration"
value = "${(var.booleanVar) == 0 ? aws_launch_configuration.default_launch_config.*.name : aws_launch_configuration.custom_launch_config.*.name}
}
Let me know if there is a better way!
You can use the same variable you used to decide which resource to enable to select the appropriate result:
output "name" {
value = var.boolean ? aws_launch_configuration.custom_launch_config[0].name : aws_launch_configuration.basic_launch_config[0].name
}
Another option, which is a little more terse but arguably also a little less clear to a future reader, is to exploit the fact that you will always have one list of zero elements and one list with one elements, like this:
output "name" {
value = concat(
aws_launch_configuration.basic_launch_config[*].name,
aws_launch_configuration.custom_launch_config[*].name,
)[0]
}
Concatenating these two lists will always produce a single-item list due to how the count expressions are written, and so we can use [0] to take that single item and return it.
In Mule4, I need to convert json sample data in to dynamic XML format, I have tried with dataweave(2.0) field mapping, getting null values. Does anyone can help me on this?
If the question is just asking out to build up XML output from JSON input, that is a pretty open-ended question. What do you want to evaluate dynamically? You could, for example, use part of the payload to set values in the DataWeave expression.
There is a more difficult version of this question: how to dynamically evaluate DataWeave code constructed into an input string, where this string could be read from various script files, or even constructed in-line from some input data (payload, attributes, or variables).
Here is another example covered in our MuleSoft DataWeave training course at http://training.mulesoft.com.
You can use a Dynamic Evaluate component to dynamically evaluate a constructed DataWeave expression string. Here is an example that replaces the uName parameter with a dynamic value.
Also, the expression is configured to read in different script files based on some condition:
output application/json
---
do {
var choice = attributes.queryParams.script default "NO_SCRIPT"
---
if(choice == "NO_SCRIPT")
"output application/json --- {result: 'NO SCRIPT ENTERED'}"
else if(choice == "script1") vars.script1
else if (choice == "script2") vars.script2
else read(choice)
}
Here are two example scripts that substitutes values for uName and produce different types of output (XML vs. JSON).
This is script1:
output application/xml
---
root: { message: "order "
++ attributes.queryParams.orderid
++ " has been received from "
++ uName, items: payload.items}
This is script2:
output application/json
---
root: { message: "Order2 "
++ attributes.queryParams.orderid
++ " has been received from "
++ uName, items: payload.items}
Notice that this example is dangerous. It lets the web client inject any DataWeave code into the Mule flow, so this example should never be copied into production code, but it does demonstrate the ability to run any DataWeave code passed into a Mule application.
I found a website where I can look up vehicle inspections in Denmark. I need to extract some information from the page and loop through a series of license plates. Lets take this car as an example: http://selvbetjening.trafikstyrelsen.dk/Sider/resultater.aspx?Reg=as87640
Here on the left table, you can see some basic information about the vehicle. On the right, you can see a list of the inspections for this specific car. I need a script, which can check if the car has any inspections and then grab the link to each of the inspection reports. Lets take the first inspection from the example. I would like to extract the onclick text from each of the inspections.
The first inspection link would be:
location.href="/Sider/synsrapport.aspx?Inspection=18014439&Vin=VF7X1REVF72378327"
or if you could extract the inspection ID and Vin variable from the URL immediately:
Inspection ID: 18014439
Vin: VF7X1REVF72378327
Here is an example of a car which don't have any inspections yet, if you want to see what that looks like: http://selvbetjening.trafikstyrelsen.dk/Sider/resultater.aspx?Reg=as87400
Current Solution plan:
Download the HTML source code as a String in VB.net
Search the string and extract the specific parts.
Store it in a StringBuilder and upload this to my SQL server
Is this the most efficient way, or do you know of any libraries which is used to specific extract elements from a website in VB.net! Thanks!
You could use Java libraries HtmlUnit or Jsoup to webscrape the page.
Here's an example using HtmlUnit:
LogFactory.getFactory().setAttribute("org.apache.commons.logging.Log", "org.apache.commons.logging.impl.NoOpLog");
java.util.logging.Logger.getLogger("com.gargoylesoftware").setLevel(Level.OFF);
java.util.logging.Logger.getLogger("org.apache.commons.httpclient").setLevel(Level.OFF);
WebClient client = new WebClient(BrowserVersion.CHROME);
client.getOptions().setJavaScriptEnabled(true);
client.getOptions().setThrowExceptionOnScriptError(false);
client.getOptions().setThrowExceptionOnFailingStatusCode(false);
HtmlPage page = client.getPage("http://selvbetjening.trafikstyrelsen.dk/Sider/resultater.aspx?Reg=as87640");
HtmlTable inspectionsTable = (HtmlTable) page.getElementById("tblInspections");
Map<String, String> inspections = new HashMap<String, String>();
for (HtmlTableRow row: inspectionsTable.getRows()) {
String[] splitRow = row.getAttribute("onclick").split("=");
if (splitRow.length >= 4) {
String id = splitRow[2].split("&")[0];
String vin = splitRow[3].replace("\"", "");
inspections.put(id, vin);
System.out.println(id + " " + vin);
}
}