I have a simple proxy service which is required to convert an XML payload into a CSV message. For this purpose I'm using smooks mediator. Following is my proxy service.
<?xml version="1.0" encoding="UTF-8"?>
<proxy name="TestProxy" startOnLoad="true" trace="disable"
transports="http https" xmlns="http://ws.apache.org/ns/synapse">
<target>
<inSequence>
<log level="custom">
<property name="STATUS" value="TESTING PROXY SERVICE"/>
</log>
<payloadFactory media-type="xml">
<format>
<csv-set>
<csv-record number="2">
<Name>Jhone</Name>
<City>Colombo</City>
<Age>32</Age>
</csv-record>
<csv-record number="3">
<Name>Doe</Name>
<City>Jaffna</City>
<Age>32</Age>
</csv-record>
</csv-set>
</format>
<args/>
</payloadFactory>
<log level="full"/>
<smooks config-key="gov:Test/smooks/TestSmooksConfig.xml">
<input type="xml"/>
<output type="text"/>
</smooks>
<log level="custom">
<property name="STATUS" value="PROCESSED MSG******"/>
</log>
<log level="full"/>
</inSequence>
<outSequence/>
<faultSequence/>
</target>
</proxy>
Following is my Smooks configuration.
<?xml version="1.0" encoding="UTF-8"?><smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd" xmlns:csv="http://www.milyn.org/xsd/smooks/csv-1.2.xsd" xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd">
<params>
<param name="inputType">input.xml</param>
<param name="stream.filter.type">SAX</param>
<param name="input.xml" type="input.type.actived">File:/F:\Practicals\WSO2\vfs_file_listening\input-csv\b.xml</param>
</params>
<ftl:freemarker applyOnElement="#document">
<ftl:template><![CDATA["Name","City","Age"
<#list .vars["csv-set"]["csv-record"] as csv_record>
"${.vars["csv_record"]["Name"]}","${.vars["csv_record"]["City"]}","${.vars["csv_record"]["Age"]}"
</#list>]]></ftl:template>
<param name="includeFieldNames">true</param>
<param name="seperator">,</param>
<param name="quote">"</param>
<param name="csvFields">Name,City,Age</param>
<param name="messageType">CSV</param>
<param name="templateDataProvider">input</param>
</ftl:freemarker>
<resource-config selector="#document">
<resource>org.milyn.delivery.DomModelCreator</resource>
</resource-config>
</smooks-resource-list>
The output message only prints the following.
<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://www.w3.org/2003/05/soap-envelope">
<soapenv:Body>
<ax:text xmlns:ax="http://ws.apache.org/commons/ns/payload">"Name","City","Age"</ax:text>
</soapenv:Body>
</soapenv:Envelope>
Not with the data in the input XML tags, Can anyone point out what I'm doing wrong?
Thanks in advance.
I was able to achieve this by the following Smooks configuration. (with the help of wso2 dev support, thanks Evanthika).
<?xml version="1.0" encoding="UTF-8"?>
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.1.xsd"
xmlns:ftl="http://www.milyn.org/xsd/smooks/freemarker-1.1.xsd">
<params>
<param name="stream.filter.type">SAX</param>
<param name="inputType">input.xml</param>
<param name="input.xml" type="input.type.actived">File:/C:\Work\2016\09_ESB_Auto_Mail_rpts\TestFiles\TestFileXMLCSVMap.xml
</param>
<param name="default.serialization.on">true</param>
</params>
<resource-config selector="csv-set,csv-record">
<resource>org.milyn.delivery.DomModelCreator</resource>
</resource-config>
<ftl:freemarker applyOnElement="csv-record">
<ftl:template>
<!-- <#ftl ns_prefixes={"D":"http://ws.apache.org/ns/synapse"}> ${.vars["csv-record"].Name},${.vars["csv-record"].City},${.vars["csv-record"].Age} -->
</ftl:template>
<param name="quote">"</param>
<param name="includeFieldNames">true</param>
<param name="csvFields">Name,City,Age</param>
<param name="seperator">,</param>
<param name="messageType">CSV</param>
<param name="templateDataProvider">input</param>
</ftl:freemarker>
</smooks-resource-list>
Changes that have been done are,
<ftl:template> with xml namespace. (In my case it comes from a dataservice, hence the http://ws.apache.org/ns/synapse namespace in the <#ftl ns_prefixes> tag).
<resource-config selector="csv-set,csv-record"> and
<ftl:freemarker applyOnElement="csv-record"> to point out the
elements for smooks to work on.
Hope this helps as reference to anyone.
Related
I read through the following documentation but cannot figure out what other configuration I need: https://github.com/logstash/logstash-logback-encoder#header-fields
My config file:
<include resource="org/springframework/boot/logging/logback/defaults.xml" />
<include resource="org/springframework/boot/logging/logback/console-appender.xml" />
<property name="LOG_FILE" value="${LOG_FILE:-${LOG_PATH:-${LOG_TEMP:-${java.io.tmpdir:-/tmp}}}/spring.log}"/>
<appender name="FILE" class="ch.qos.logback.core.rolling.RollingFileAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder">
<customFields>{"app_name":"${APP_NAME:-N/A}","app_version":"${APP_VERSION:-N/A}","hostname":"${HOST:-N/A}","environment":"${environment:-${ENVIRONMENT:-N/A}}"}</customFields>
<includeContext>false</includeContext>
<timeZone>UTC</timeZone>
</encoder>
<encoder class="net.logstash.logback.encoder.LogstashAccessEncoder">
<fieldNames>
<requestHeaders>request_headers</requestHeaders>
</fieldNames>
</encoder>
<file>${LOG_FILE}</file>
<rollingPolicy class="ch.qos.logback.core.rolling.SizeAndTimeBasedRollingPolicy">
<fileNamePattern>${LOG_FILE_ROTATION:-${LOG_FILE}%d{yyyy-MM-dd}.%i}</fileNamePattern>
<maxHistory>${LOG_FILE_ROTATION_MAX_HISTORY:-1}</maxHistory>
<totalSizeCap>${LOG_FILE_ROTATION_TOTAL_SIZE_CAP:-3GB}</totalSizeCap>
<maxFileSize>${LOG_FILE_ROTATION_MAX_FILE_SIZE:-1GB}</maxFileSize>
</rollingPolicy>
</appender>
<root level="${LOG_LEVEL:-INFO}">
<appender-ref ref="CONSOLE" />
<appender-ref ref="FILE" />
</root>
</configuration>
Error:
java.lang.IllegalStateException: Logback configuration error detected:
ERROR in ch.qos.logback.core.joran.spi.Interpreter#19:29 - no applicable action for [requestHeaders], current ElementPath is [[configuration][appender][encoder][fieldNames][requestHeaders]]
The xml configuration file you provided appears to be for logback-classic (since it includes <root level=...).
Automatic logging of request headers by logstash-logback-encoder is only available for IAccessEvents logged via logback-access.
In other words, automatic logging of request headers is not available for ILoggingEvents logged via a Logger from logback-classic. However, you can manually include them when using logback-classic as described below.
Logging of request headers with logback-access
To log request headers for IAccessEvents logged via logback-access, follow the instructions for setting up logback-access for tomcat or jetty, and add the following to your logback-access.xml. (Note this is not the logback.xml file used by logback-classic)
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashAccessEncoder">
<fieldNames>
<requestHeaders>request_headers</requestHeaders>
</fieldNames>
</encoder>
</appender>
<appender-ref ref="console"/>
</configuration>
This configuration was tested using logstash-logback-encoder 5.3. Note that the xml element names are different in logstash-logback-encoder versions prior to 5.0, so ensure you are using 5.0+ with that configuration.
Logging of request headers with logback-classic
logstash-logback-encoder does not provide first-class support for logging request headers via logback-classic. However, you include them in a log event logged via a Logger by using event-specific custom fields.
For example, in a class that has access to the http request (such as a servlet filter), you could do something like this:
Map<String, String> httpHeadersMap = ...; // get http request headers as a map
LOGGER.info("request", StructuredArguments.entries(httpHeadersMap));
and configure a LogbackEncoder in your logback.xml like this:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<configuration>
<appender name="console" class="ch.qos.logback.core.ConsoleAppender">
<encoder class="net.logstash.logback.encoder.LogstashEncoder"/>
</appender>
<root level="info">
<appender-ref ref="console" />
</root>
</configuration>
i'm facing some problems with this process.. let me explain:
i want to call two services and join the responses in this order:
i have the first service's response:
<locations xmlns="http://ssocial.com/traceit/services/location">
<location>
<latitude>-33.45209980</latitude>
<longitude>-70.66241990</longitude>
<altitude>0.0</altitude>
<creation>2016-09-04T14:59:48.000-03:00</creation>
<updated>2016-09-04T14:59:48.000-03:00</updated>
<deviceId>25</deviceId>
</location>
<location>
<latitude>-33.45149521</latitude>
<longitude>-70.66146224</longitude>
<altitude>817.0</altitude>
<creation>2016-09-04T13:18:43.000-03:00</creation>
<updated>2016-09-04T13:18:43.000-03:00</updated>
<deviceId>25</deviceId>
</location>
</locations>
then i have the secon service's response :
<device xmlns="http://ssocial.com/traceit/services/device">
<id>25</id>
<code>mobile_02</code>
<type>
<id>1</id>
<name>MOBILE</name>
</type>
And then, the final response that i would like to have is :
<locations xmlns="http://ssocial.com/traceit/services/location">
<location>
<latitude>-33.45209980</latitude>
<longitude>-70.66241990</longitude>
<altitude>0.0</altitude>
<creation>2016-09-04T14:59:48.000-03:00</creation>
<updated>2016-09-04T14:59:48.000-03:00</updated>
<deviceId>25</deviceId>
<device xmlns="http://ssocial.com/traceit/services/device">
<id>25</id>
<code>mobile_02</code>
<type>
<id>1</id>
<name>MOBILE</name>
</type>
</device>
</location>
<location>
<latitude>-33.45149521</latitude>
<longitude>-70.66146224</longitude>
<altitude>817.0</altitude>
<creation>2016-09-04T13:18:43.000-03:00</creation>
<updated>2016-09-04T13:18:43.000-03:00</updated>
<deviceId>25</deviceId>
<device xmlns="http://ssocial.com/traceit/services/device">
<id>25</id>
<code>mobile_02</code>
<type>
<id>1</id>
<name>MOBILE</name>
</type>
</device>
</location>
</locations>
i'm really having lot of troubles trying to join both responses like that, and this is my sequence in wso2:
<sequence name="getDeviceInfoByLocationProcess" trace="disable" xmlns="http://ws.apache.org/ns/synapse">
<property expression="count(//ns1:locations/ns1:location)" name="LOCATIONS_COUNT" scope="default" type="STRING" xmlns:ns1="http://ssocial.com/traceit/services/location"/>
<iterate description="" expression="//ns1:locations/ns1:location/ns1:deviceId" id="LOCATIONS_ITERATOR" xmlns:ns1="http://ssocial.com/traceit/services/location">
<target>
<sequence>
<property description="device_id" expression="//ns1:deviceId" name="device_id" scope="default" type="STRING"/>
<sequence key="getDeviceInfoProcess"/>
</sequence>
</target>
</iterate>
<log level="custom">
<property expression="//ns1:locations" name="LOCATIONS" xmlns:ns1="http://ssocial.com/traceit/services/location"/>
<property expression="//ns2:device" name="DEVICE_INFO" xmlns:ns2="http://ssocial.com/traceit/services/device"/>
</log>
<property name="ECNCLOSING_ELEMENT" scope="default">
<wrapper xmlns=""/>
</property>
<aggregate id="AGG_01">
<completeCondition>
<messageCount max="-1" min="{get-property('LOCATIONS_COUNT')}"/>
</completeCondition>
<onComplete expression="//ns2:device" xmlns:ns2="http://ssocial.com/traceit/services/device">
<log level="full">
<property expression="$ctx:LOCATIONS_COUNT" name="COUNTER"/>
<property expression="$ctx:ECNCLOSING_ELEMENT" name="ENCLOSING"/>
</log>
<call/>
</onComplete>
</aggregate>
and actually with this sequence, i'm just getting this:
<device xmlns="http://ssocial.com/traceit/services/device">
<id>25</id>
<code>mobile_02</code>
<type>
<id>1</id>
<name>MOBILE</name>
</type>
In advance , thank you very much for the help
I can't see the content of "getDeviceInfoProcess" sequence, but I guess your are using call mediator. So, as a result of those requests (multiple requests because of iterate mediator), you receive some responses and that what you have to deal with in the rest of the initial mediation : aggregate offers you a way to wait for all the responses, but the initial message is lost : you should save it in a property (type OM) and use something like payloadMediator inside aggregate/onComplete to compose the final message.
Very simple example. Request:
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
<soapenv:Header/>
<soapenv:Body>
<T>
<m1>bysy</m1>
<m1>momo</m1>
<m1>gigi</m1>
</T>
</soapenv:Body>
</soapenv:Envelope>
I want to iterate and log every element in tag m1.
Proxy code:
<proxy xmlns="http://ws.apache.org/ns/synapse" name="Loopback" transports="https,http"
<target>
<inSequence>
<property name="OUT_ONLY" value="true" scope="axis2"/>
<iterate expression="//T/m1">
<target>
<sequence>
<property name="OUT_ONLY" value="true" scope="axis2"/>
<log level="custom">
<property name="M1" expression="//text()"/>
</log>
<drop/>
</sequence>
</target>
</iterate>
</inSequence>
</target>
</proxy>
It work as expected and logs all values. But WSO2ESB server doesn't close connection, until it falls with timeout. (I use SOAPUI for testing)
I tried to put properties OUT_ONLY RESPOND NO_KEEPALIVE in different part of proxy but it didn't help, as well I tried to use , with same result.
It looks like problem in iterate mediator which doesn't let to inSequence to terminate and send response to client. Once i remove it, proxy disconnect immediately with status code '202' message accepted.
This is because, by default, attribute "continueParent" is set to false with iterate mediator : change to <iterate continueParent="true" expression="//T/m1">
And you should explicitly send a 202 inside your inSequence, set this property : <property name="FORCE_SC_ACCEPTED" value="true" scope="axis2"/>
I'm using wso2dss 3.1.0 and wso2esb 4.7.0.I wish to fire select from esb.I have write select query in dss and dbs for this is like :
<query id="Capp_select_emercontactid" useConfig="default">
<sql>select userid,mailid,phonenumber from muser where phonenumber = ? or mailid = ?</sql>
<result element="Entries" rowName="Entry">
<element column="userid" name="userid" xsdType="string"/>
<element column="mailid" name="mailid" xsdType="string"/>
<element column="phonenumber" name="phonenumber" xsdType="string"/>
</result>
<param name="phonenumber" ordinal="1" sqlType="STRING"/>
<param name="mailid" ordinal="2" sqlType="STRING"/>
</query>
<operation disableStreaming="true" name="Capp_select_emercontactid_op">
<call-query href="Capp_select_emercontactid">
<with-param name="phonenumber" query-param="phonenumber"/>
<with-param name="mailid" query-param="mailid"/>
</call-query>
</operation>
It's working fine in dss.
Now i have write a payload in esb like :
<payloadFactory>
<format>
<p:Capp_select_emercontactid_op xmlns:p="http://ws.wso2.org/dataservice">
<p:phonenumber>$1</p:phonenumber>
<p:mailid>$2</p:mailid>
</p:Capp_select_emercontactid_op>
</format>
<args>
<arg expression="get-property('phoneno1')" evaluator="xml"/>
<arg expression="get-property('mailid1')" evaluator="xml"/>
</args>
</payloadFactory>
<log level="full"/>
<send>
<endpoint>
<address uri="http://localhost:9764/services/Capp_MuserDataservice/" format="soap11"/>
</endpoint>
</send>
When i'm hiting this proxy it didn't give responce..Why so?Is it correct configuration?Please let me know
I never had any luck using the send mediator with the DSS. Could you consider using the CalloutMediator instead (it's a synchronous send/response call so you may need to use a respond mediator after if you're wanting an immediate response to your client).
It'd look something like:
<callout xmlns="http://ws.apache.org/ns/synapse" action="Capp_select_emercontactid_op">
<source xmlns:p="http://ws.wso2.org/dataservice" xpath="//p:Capp_select_emercontactid_op"/>
<target xmlns:p="http://ws.wso2.org/dataservice" xpath="//p:Capp_select_emercontactid_op"/>
</callout>
I am executing a sample given for VFS file processing, but during execution getting the error as "Cannot find the object for smooks config key: smooks".
Could you please let me know what I have missed out?
Configurations made are as below.
Local entry for smooke is created with below configuration in smooke-key.xml
<localEntry key="smooke" src="file:/development/Dev/wso2esb-4.8.0/repository/resources/smooks-config.xml"><description/></localEntry>
==================================================================================
Smooke-config.xml is as below
<smooks-resource-list xmlns="http://www.milyn.org/xsd/smooks-1.0.xsd">
<!--Configure the CSVParser to parse the message into a stream of SAX events. -->
<resource-config selector="org.xml.sax.driver">
<resource>org.milyn.csv.CSVParser</resource>
<param name="fields" type="string-list">name,value</param>
</resource-config>
Proxy service is as below
<?xml version="1.0" encoding="UTF-8"?>
<proxy xmlns="http://ws.apache.org/ns/synapse"
name="FileProcessor"
transports="vfs"
startOnLoad="true"
trace="disable">
<description/>
<target>
<inSequence>
<log level="full"/>
<smooks config-key="smooke">
<input type="text"/>
<output type="xml"/>
</smooks>
<clone>
<target sequence="fileWriteSequence"/>
<target sequence="databaseSequence"/>
</clone>
</inSequence>
</target>
<parameter name="transport.vfs.ActionAfterProcess">MOVE</parameter>
<parameter name="transport.PollInterval">15</parameter>
<parameter name="transport.vfs.MoveAfterProcess">file:///development/Dev/fileProcessing/sourcefiles</parameter>
<parameter name="transport.vfs.FileURI">file:///development/Dev/fileProcessing/in</parameter>
<parameter name="transport.vfs.MoveAfterFailure">file:///development/Dev/fileProcessing/error</parameter>
<parameter name="transport.vfs.FileNamePattern">.*.txt</parameter>
<parameter name="transport.vfs.ContentType">text/plain</parameter>
<parameter name="transport.vfs.ActionAfterFailure">MOVE</parameter>
</proxy>
create the smooks configuration at
wso2esb-4.8.0/repository/samples/resources/smooks/Smooke-config.xml
and restart the esb
change the localentry file like this
<localEntry key="smooke" src="file:repository/resources/smooks-config.xml"><description/></localEntry>