Apache Camel select then insert - sql

I am very very new to Apache Camel and I am looking to create a simple application which reads 200 products from mysql db 1 and then inserts the products into mysql db 2. No changes required to the data, it is just a one for one bulk select and then bulk insert.
I have been looking at the camel-sql-example project which uses the sql component. I like the way it's structured so I've attempted to go with this example project as my base.
Would anyone be able to help me with the location of a good example for this or giving me a rough indication of what my camel route (spring arcehtype) should look like?
Is something like this correct?
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="
http://camel.apache.org/schema/spring http://camel.apache.org/schema/spring/camel-spring.xsd
http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans.xsd">
<bean id="sourceDataSource" class="org.apache.commons.dbcp.BasicDataSource">
...
</bean>
<bean id="targetDataSource" class="org.apache.commons.dbcp.BasicDataSource">
...
</bean>
<!-- configure the Camel SQL component to use the JDBC data source -->
<bean id="sourceSql" class="org.apache.camel.component.sql.SqlComponent">
<property name="dataSource" ref="sourceDataSource"/>
</bean>
<bean id="targetSql" class="org.apache.camel.component.sql.SqlComponent">
<property name="dataSource" ref="targetDataSource"/>
</bean>
<bean id="productBean" class="org.apache.camel.example.sql.ProductBean"/>
<camelContext xmlns="http://camel.apache.org/schema/spring">
<propertyPlaceholder location="classpath:sql.properties" id="placeholder"/>
<route id="processProduct-route">
<from uri="sourceSql:{{sql.selectProduct}}"/>
<to uri="targetSql:{{sql.insertProduct}}"/>
<log message="${body}"/>
</route>
</camelContext>
thanks

I tested this today and what I've specified above seems to work! Something that I did not expect though was that the route just loops and continues to run multiple times. I only need it to run once. I'm looking into how to achieve this.

Related

Session.removeAttribute is not working as expected

Sample project showing issue
What I'm noticing is that session.removeAttribute doesn't really remove the attribute, it only sets the value to null.
I'm not sure if this is a setup issue on my end or what.
I thought this might be related to jsf, removed jsf and the issue persists.
Really want to use ignite but am struggling to get this working correctly.
Any input would be helpful, has anyone else seen this?
Thanks in advance, could use some help! If anyone has any idea's please let me know and thank you.
dshaw
The closest issue that I could find to my particular issue was:
https://github.com/apache/ignite/pull/2243
I patched my local ignite repo with #2243 , redeployed to tomcat and 2
ignite nodes but am seeing the same issue.
My setup (client/server):
Ignite 2.5.0 (2 node cluster)
Apache Tomcat 7
Java 9
tomcat 7 setenv.sh
#!/bin/sh
export JAVA_HOME=/opt/java/jdk-10.0.1
export PATH=$JAVA_HOME/bin:$PATH
export CATALINA_OPTS="--add-exports java.base/jdk.internal.misc=ALL-UNNAMED
--add-exports java.base/sun.nio.ch=ALL-UNNAMED
--add-exports java.management/com.sun.jmx.mbeanserver=ALL-UNNAMED
--add-exports jdk.internal.jvmstat/sun.jvmstat.monitor=ALL-UNNAMED
--add-modules java.xml.bind"
export CATALINA_HOME=/opt/apache/apache-tomcat-7.0.86_node
Config use by 3 tomcat nodes
<?xml version="1.0" encoding="UTF-8"?>
<!--
Licensed to the Apache Software Foundation (ASF) under one or more
contributor license agreements. See the NOTICE file distributed with
this work for additional information regarding copyright ownership.
The ASF licenses this file to You under the Apache License, Version 2.0
(the "License"); you may not use this file except in compliance with
the License. You may obtain a copy of the License at
http://www.apache.org/licenses/LICENSE-2.0
Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
-->
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<!--
Alter configuration below as needed.
-->
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<property name="clientMode" value="true"/>
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>172.24.2.156:47500..47509</value>
<value>172.24.3.28:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
tomcat web.xml snip
<listener>
<listener-class>org.apache.ignite.startup.servlet.ServletContextListenerStartup</listener-class>
</listener>
<filter>
<filter-name>IgniteWebSessionsFilter</filter-name>
<filter-class>org.apache.ignite.cache.websession.WebSessionFilter</filter-class>
</filter>
<!-- You can also specify a custom URL pattern. -->
<filter-mapping>
<filter-name>IgniteWebSessionsFilter</filter-name>
<url-pattern>/*</url-pattern>
</filter-mapping>
<!-- Specify Ignite configuration (relative to META-INF folder or Ignite_HOME). -->
<context-param>
<param-name>IgniteConfigurationFilePath</param-name>
<param-value>config/client-config.xml</param-value>
</context-param>
<!-- Specify the name of Ignite cache for web sessions. -->
<context-param>
<param-name>IgniteWebSessionsCacheName</param-name>
<param-value>medata-replicated</param-value>
</context-param>
Config used by 2 ignite nodes
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<!--
Alter configuration below as needed.
-->
<bean id="grid.cfg" class="org.apache.ignite.configuration.IgniteConfiguration">
<!-- Enable cache events. -->
<property name="includeEventTypes">
<util:constant static-field="org.apache.ignite.events.EventType.EVTS_CACHE"/>
</property>
<property name="cacheConfiguration">
<list>
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<!-- Cache name. -->
<property name="name" value="medata-replicated"/>
<!-- Cache mode. -->
<property name="cacheMode" value="REPLICATED"/>
</bean>
</list>
</property>
<property name="discoverySpi">
<bean class="org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi">
<property name="ipFinder">
<bean class="org.apache.ignite.spi.discovery.tcp.ipfinder.vm.TcpDiscoveryVmIpFinder">
<property name="addresses">
<list>
<value>172.24.2.156:47500..47509</value>
<value>172.24.3.28:47500..47509</value>
</list>
</property>
</bean>
</property>
</bean>
</property>
</bean>
</beans>
This issue is also discussed on Apache Ignite user forum: http://apache-ignite-users.70518.x6.nabble.com/Session-removeAttribute-is-not-working-as-expected-td21762.html

Multiple Persistence Store for Apache Ignite

I have one use case where I have to support multiple persistence store for my ignite cluster,For example Cache A1 should be primed from Database db1 and Cache B1 should be primed from database db2. can this be done?.In ignite Configuration XML I can only provide one persistence store details,
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:util="http://www.springframework.org/schema/util"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans.xsd
http://www.springframework.org/schema/util
http://www.springframework.org/schema/util/spring-util.xsd">
<!-- Datasource for Persistence. -->
<bean name="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="oracle.jdbc.driver.OracleDriver" />
<property name="url" value="jdbc:oracle:thin:#localhost:1521:roc12c" />
<property name="username" value="test" />
<property name="password" value="test" />
</bean>
In my CacheStore implementation I can only access this Database right?.
I've not tried this, but if its similar to other bean-configured systems. You should be able to create another bean with a different name and configuration. Then in your cache configuration for A1 and B1 specify the different data sources. That being said, I'm guessing that theoretically.
It may be that you are already doing so, but I can't tell from your question. If you instead choose to implement your caches in this manner https://apacheignite.readme.io/docs/persistent-store you can definitely configure two caches to have different data sources. This is how I'm currently implementing multiple caches. In the cache store I use I specifically call out which database to go to.
Here is a cache configuration I use for mine.
<property name="cacheConfiguration">
<bean class="org.apache.ignite.configuration.CacheConfiguration">
<!-- Set a cache name. -->
<property name="name" value="recordData"/>
<property name="rebalanceMode" value="ASYNC"/>
<property name="cacheMode" value="PARTITIONED"/>
<property name="backups" value="1"/>
<!-- Enable Off-Heap memory with max size of 10 Gigabytes (0 for unlimited). -->
<property name="memoryMode" value="OFFHEAP_TIERED"/>
<property name="offHeapMaxMemory" value="0"/>
<property name="swapEnabled" value="false"/>
<property name="cacheStoreFactory">
<bean class="javax.cache.configuration.FactoryBuilder" factory-method="factoryOf">
<constructor-arg value="com.company.util.MyDataStore"/>
</bean>
</property>
<property name="readThrough" value="true"/>
<property name="writeThrough" value="true"/>
</bean>
</property>
Cache store is configured per cache, so you just need to inject different data sources to different stores. What you showed is just a standalone data source bean, it's not even a part of IgniteConfiguration. You can have multiple data source beans with different IDs.

How to use simple-spring-memcached with AWS Auto Discovery

How to use simple-spring-memcached library (SSM) with AWS Elasti Cache Auto Discovery feature? We are using spymemcached as client.
So currently you are using spymemcached and want to add cache layer using simple spring memcached (SSM), right? If yes then please provide your current configuration for spymemcached. It should be easy to use the same configuration with SSM.
UPDATE
I've added new dedicated memcached provider in SSM that uses AWS ElastiCache Cluster Client. It is available on master branch and not released yet. If you build SSM from master or use snapshot available in this repository then you can use Auto Discovery feature.
Remove dependencies to spymemcached-provider and spymemcached instead add a new dependency:
<dependency>
<groupId>com.google.code.simple-spring-memcached</groupId>
<artifactId>aws-elasticache-provider</artifactId>
<version>3.4.1-SNAPSHOT</version>
</dependency>
Use below configuration:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xmlns:context="http://www.springframework.org/schema/context" xmlns:cache="http://www.springframework.org/schema/cache"
xsi:schemaLocation="
http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.1.xsd
http://www.springframework.org/schema/context
http://www.springframework.org/schema/context/spring-context-3.1.xsd
http://www.springframework.org/schema/cache
http://www.springframework.org/schema/cache/spring-cache-3.1.xsd">
<cache:annotation-driven />
<bean name="cacheManager" class="com.google.code.ssm.spring.SSMCacheManager">
<property name="caches">
<set>
<bean class="com.google.code.ssm.spring.SSMCache">
<constructor-arg name="cache" index="0" ref="defaultCache" />
<!-- 5 minutes -->
<constructor-arg name="expiration" index="1" value="300" />
<!-- #CacheEvict(..., "allEntries" = true) won't work because allowClear is false,
so we won't flush accidentally all entries from memcached instance -->
<constructor-arg name="allowClear" index="2" value="false" />
</bean>
</set>
</property>
</bean>
<bean name="defaultCache" class="com.google.code.ssm.CacheFactory">
<property name="cacheName" value="defaultCache" />
<property name="cacheClientFactory">
<bean name="cacheClientFactory" class="com.google.code.ssm.providers.elasticache.MemcacheClientFactoryImpl" />
</property>
<property name="addressProvider">
<bean class="com.google.code.ssm.config.DefaultAddressProvider">
<!-- set only single address to configuration endpoint -->
<property name="address" value="mycluster.fnjyzo.cfg.use1.cache.amazonaws.com:11211" />
</bean>
</property>
<property name="configuration">
<bean class="com.google.code.ssm.providers.elasticache.ElastiCacheConfiguration">
<!-- set client mode to dynamic to enable Auto Discovery feature -->
<property name="clientMode" value="#{T(net.spy.memcached.ClientMode).Dynamic}" />
</bean>
</property>
</bean>
</beans>
And let me know if it works for you.
UPDATE 2
New Simple Spring Memcached version 3.5.0 with AWS Auto Discovery feature is available on github and central maven repository.

Mule - make Global Elements more global

In mule I have many applications running on the same container that access a jdbc connector with the same connection string/user/password set.
Of course any app has configured the same global connector in its xml configuration file, so there is code duplication.
Is there a way to define only once per container the connection and access it from any app?
I would try this: have one app create the datasource and store it in JNDI and have the other apps pick it up from JNDI.
Since there is no strong guarantee of app start ordering, it's possible that one app that needs the JNDI datasource would start too soon. You would need to configure Spring to be able to perform the JNDI lookup again in case of failure and configure a threaded retry policy on the Mule JDBC connector.
Also you will need to install the datasource and database JARs in lib/user so all apps could use them.
Just create a spring bean for your JDBC connector in xml some where in your system and have all your applications load it in your apps:
<spring:import resource="JDBC-beans.xml" />
and the xml:
<?xml version="1.0" encoding="UTF-8"?>
<beans xmlns="http://www.springframework.org/schema/beans"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans
http://www.springframework.org/schema/beans/spring-beans-3.0.xsd ">
<!-- Initialization for data source -->
<bean id="dataSource"
class="org.springframework.jdbc.datasource.DriverManagerDataSource">
<property name="driverClassName" value="com.mysql.jdbc.Driver"/>
<property name="url" value="jdbc:mysql://localhost:3306/TEST"/>
<property name="username" value="root"/>
<property name="password" value="password"/>
</bean>
</beans>
I have solved this kind of problem using a Domain project, where I inserted all database configurations that have been used by other projects.

Unable to set up interceptor/brokerplugin in ActiveMQ

I have written a simple authentication/authorization plugin that I want to inject into ActiveMQ. I want it to be called onConnect and onSubscribe. I have followed these steps at the ActiveMQ website but a couple of things happen.
1) If I put in my bean declaration in the default activemq.xml file in //beans/broker/plugins I get a validation error saying that the node "bean" is not allowed there.
2) If I put the plugin declaration outside of the broker element it will inject the element, but it will not call installPlugin() nor the hooks, presumably because that is for the broker to do.
3) If I change the XML namespace declaration in the default activemq.xml (http://activemq.apache.org/schema/core) to that which is stated in the docs listed above (http://activemq.org/config/1.0) along with the proper URLs, I get the error that it cannot find the schema document.
The only thing I can come up with is that either there were changes in 5.6 that were not reflected in the documentation, I'm doing something very wrong, or I'm just crazy. Here is the relevant portion of the xml doc (minus several nodes not directly related to the problem).
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.org/config/1.0"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<broker xmlns="http://activemq.apache.org/schema/core" brokerName="localhost" dataDirectory="${activemq.data}">
<plugins>
<bean id="tokenLoginPlugin" class="auth.TokenLoginPlugin">
<property name="host" value="localhost" />
</bean>
</plugins>
</broker>
This generates the following exception.
The matching wildcard is strict, but no declaration can be found for element 'broker'.
If I use the xmlns declarations from the default activemq.xml file, I get the following.
Invalid content was found starting with element 'bean'
I can see that it's a validation error, but none of the docs seem to be pointing me in the right direction.
Figured it out, though I had tried it before and it hadn't worked. Perhaps I had messed up my namespaces the last time. I changed my plugin definition and added the Spring namespace to my bean declaration.
<plugins>
<bean id="tokenLoginPlugin" class="auth.TokenLoginPlugin" xmlns="http://www.springframework.org/schema/beans">
<property name="host" value="localhost" />
</bean>
</plugins>
My configuration is :
<beans
xmlns="http://www.springframework.org/schema/beans"
xmlns:amq="http://activemq.apache.org/schema/core"
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
xsi:schemaLocation="http://www.springframework.org/schema/beans http://www.springframework.org/schema/beans/spring-beans-2.0.xsd
http://activemq.apache.org/schema/core http://activemq.apache.org/schema/core/activemq-core.xsd">
<plugins>
<bean xmlns="http://www.springframework.org/schema/beans" id="probePlugin" class="com.ProbePlugin"/>
</plugins>
</beans>