Spring Config Server does not seem to notify Bus - spring-cloud-config

I am using Spring 2.0.1.RELEASE and have setup all projects (2 services and the cloud config server) with spring-cloud-bus
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
The config server also has the spring-cloud-config-monitor
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-config-monitor</artifactId>
</dependency>
I edit a file in my Git reposiroty (using local files with native profile of Spring Cloud Config). The change is detected, and I see the following line in the
Cloud Config Server:
17:59:25.201 [task-scheduler-3] INFO o.s.cloud.bus.event.RefreshListener - Received remote refresh request. Keys refreshed [version.client.min]
However, none of the other services receive the notification about updated keys.
On the other hand, if I manually call the bus-refresh endpoint of any other service, I see that all modules receive the updated key. The config server itself also receives the notification, but it says that there is no key updated, which makes sense since it already detected the change.
The documentation did not mention any special property to set apart from the RabbitMQ properties (which seem to be well configured since the bus-refresh endpoint is working as expected.)
I saw that there are already a few posts about this, one is even pointing to a bug that has been marked as resolved (https://github.com/spring-cloud/spring-cloud-bus/issues/101) but it does not seem to be working on my side.
Any property to enable for the config server to notify the bus?
Any hint regarding how to debug this?

Easy fix (after a lots of research!)
Changed all dependencies of org.springframework.cloud from FINCHLEY.M9 to 2.0.0.RC1 and suddenly, everything started working!

Perhaps your bootstrap.properties file aren't getting loaded on project startup
So a few things firsthand:
If you're using, (in your Cloud Config project),
<spring-cloud.version> 2020.0.0 (to be found under <DependencyManagement> or either specified in <properties>)
Then Spring Boot versions lower then 2.4.1 (in the same project) will not start up the cloud config server under its default dependencies.
So if you áre using the above versions, and maybe versions above,
Then for the projects that need updating by cloud Bus (using bootstrap.properties for instance) should contain the Starter Bootstrap dependency (of course along with the cloud-starter-config and the bus-amqp dependency)
<!-- Spring Cloud Config + Cloud Bus + Bootstrap-->
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-config</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bus-amqp</artifactId>
</dependency>
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-bootstrap</artifactId>
</dependency>
Plus check that the Spring cloud version is 2020.0.0-M6 or Hoxton.BUILD-SNAPSHOT depending on which Spring Boot version you're using. Here's a screenshot of which Spring Cloud versions are compatible with which Spring Boot versions
v v v v v v v

Related

S3 bucket file upload with persistable transfer listener

I am working on a feature to upload/resume/pause files to amazon S3 bucket and using following dependency (among others) in pom.xml for these features,
<dependency>
<groupId>software.amazon.awssdk</groupId>
<artifactId>s3-transfer-manager</artifactId>
<version>2.19.1</version>
</dependency>
I came across this link (though its for aws android sdk), it mentions adding listener to persist the state transferManager.upload(putRequest, new S3ProgressListener() {
I am using S3TransferManager in my implementation and thought of adding transfer listener to UploadFileRequest.Builder to achieve whats mentioned in first link but could not find any references to do so. Are there any references for this? Please advise.

Reading extra properties or host addresses inside the JSON test case

I am using zerocode-rest-bdd maven lib with following version. I have my applications host and port defined in the JUnit runner with "#TargetEnv("app_host.properties")".
<dependency>
<groupId>org.jsmart</groupId>
<artifactId>zerocode-rest-bdd</artifactId>
<version>1.2.15</version>
</dependency>
I want to access more hosts/ports(boundary application IPs) and common tokens(SAML, OAuth etc) into my JSON test case which I am unable to access using ${{app_host}}.
Is there any other way to extend or configure these extra properties so that I can access them and validate my boundary contracts?
app_host.properties contains:
web.application.endpoint.host=https://api.github.local
web.application.endpoint.port=443
web.application.endpoint.context=
#Can not access these below properties
app_host_1=https://app1.host.local.uk
saml_token=<SAML>sdf-wer</SAML>
Accessing in the test case as below:
"url": "${app_host_1}/users/u123",
Seems like you are using an older version of Zerocode lib. You can update to <version>1.2.17</version> or to the latest as below.
<dependency>
<groupId>org.jsmart</groupId>
<artifactId>zerocode-tdd</artifactId>
<version>1.3.x</version>
</dependency>
Then, once you define your properties e.g.
micro_host_1=https://micro.host1.local.uk
xyz_key=abc_value
the values can be resolved via ${micro_host_1} or ${xyz_key} to, https://micro.host1.local.uk or abc_value respectively in the test cases.
Hope this helps!

Connect MobileFirst Application with sql database

I have made new application Mobilefirst 8.0 using CLI.
I have followed Link to create sql adapter.
Added SQL databse file in root folder of project (root folder/Utils) is this right coz in 7.1 we have to add sql file under server folder.
Also added jdbc lib file in root folder. But when I trying to invoke adapter I am getting "Exception was thrown while invoking procedure: getAccountTransactions2 in adapter: SampleAdapter SQL connection creation failed" in logs.
Can someone let me know what I am doing wrong. Below is my code uploaded in drive.
Code here
Edit: I see what you did... you got it all wrong. You cannot include a "database file" in the application and expect the adapter to "connect" to this database.
Here is a diagram:
[application] ----> [mobilefirst server][adapter] ---> [database].
The application sends a request to the server to call the adapter which will send a request to the database, and then the response propagates back until it reaches the application, which send the original request.
You need to run your database in an actual server, not in the application.
Added SQL databse file in root folder of project (root folder/Utils) is this right coz in 7.1 we have to add sql file under server folder.
I assume you are referring to the connector driver... This is wrong.
In v8.0 you add the connector as a Maven dependency in the adapter's pom.xml file.
Learn about maven dependencies:
https://mobilefirstplatform.ibmcloud.com/tutorials/en/foundation/8.0/adapters/creating-adapters/#dependencies
https://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html
Depending on your database type, search for a connector in the maven repository site: http://search.maven.org/
Once you found it, add its reference to the pom.xml file and re-build your adapter.
Be sure though to add the correct values for your database, in the adapter.xml file (URL to database, username, password, ...).
For example for MySQL:
pom.xml
<dependencies>
<dependency>
<groupId>com.ibm.mfp</groupId>
<artifactId>adapter-maven-api</artifactId>
<scope>provided</scope>
<version>[8.0.0,9.0.0)</version>
</dependency>
<dependency>
<groupId>mysql</groupId>
<artifactId>mysql-connector-java</artifactId>
<version>5.1.6</version>
</dependency>
</dependencies>
adapter.xml
...
...
<dataSourceDefinition>
<driverClass>com.mysql.jdbc.Driver</driverClass>
<url>jdbc:mysql://localhost:3306/mobilefirst_training</url>
<user>mobilefirst</user>
<password>mobilefirst</password>
</dataSourceDefinition>

java.lang.NoSuchMethodError: org.apache.xml.security.encryption.XMLCipher.martial(Document;ReferenceList;)

I m trying to encrypt my XMl with this call:
XMlCipher.martial(Document context,ReferenceList referenceList)
for this the Project is referencing the xmlsec-1.5.3.jar
I'm able to build and deploy the code successfully on weblogic server , but post execution I'm getting the below error in logs.
Caused by: java.lang.NoSuchMethodError:
org.apache.xml.security.encryption.XMLCipher.martial(Lorg/w3c/dom/Document;Lorg/apache/xml/security/encryption/ReferenceList;)Lorg/w3c/dom/Element;
at com.ally.util.encryption.EncryptionUtil.generateEncryptedXML(EncryptionUtil.java:381)
at com.ally.util.encryption.EncryptionUtil.getEnryptedXMLDocument(EncryptionUtil.java:443)
at com.ally.partner.fis.acctinq.FISSoapHandler.getFISEncryptedHeader(FISSoapHandler.java:154)
This kind of error is ALWAYS caused by having more than one reference of the XMlCipher on your classpath. It could be a duplicate xmlsec.jar or another instance of the XMlCipher in a different jar.
You have a few options to debug the problem:
Edit your war file to remove the duplicate jar and/or different version of the XMlCipher class
Connect to your weblogic managed server with jconsole and look at the classpath. See if you notice another instance of the xmlsec.jar being included
You may need to tell weblogic to prefer the instance of the class that lives within your .war/.ear file by specifying the following in your weblogic.xml:
<container-descriptor>
<prefer-web-inf-classes>true</prefer-web-inf-classes>
</container-descriptor>
<wls:container-descriptor>
<wls:prefer-application-packages>
<wls:package-name>org.apache.xml.security.*</wls:package-name>
</wls:prefer-application-packages>
</wls:container-descriptor>
See the docs for more details and similar questions like: Weblogic 10.3.1.0 is using com.bea.core.apache.commons.net_1.0.0.0_1-4-1.jar... I want to use commons-net-2.0.jar from my code
I realize you didn't mention the use of Maven or SoapUI, but I was receiving a similar exception (java.lang.NoSuchMethodError: org.apache.xml.security.encryption.XMLCipher.setSecureValidation) when attempting to use the soapui-maven-plugin and wanted to post my solution to hopefully help.
The fix for me was to add an xml-security dependency to the soapui-maven-plugin inside the pom.xml like so:
<!-- To run, enter this command from the module directory: mvn soapui:test -->
<build>
<plugins>
<plugin>
<groupId>com.smartbear.soapui</groupId>
<artifactId>soapui-maven-plugin</artifactId>
<version>5.1.1</version>
<dependencies>
<dependency>
<groupId>org.apache.santuario</groupId>
<artifactId>xmlsec</artifactId>
<version>1.5.2</version>
</dependency>
</dependencies>
<configuration>
<projectFile>src/test/soap-ui/your-soapui-xml-file.xml</projectFile>
</configuration>
</plugin>
</plugins>
In my case, I didn't have an xml security dependency anywhere else in my project.
In other cases, it seems like some may be experiencing this error due to having multiple xml security dependencies throughout their project (and they are conflicting, like one answer is already alluding to). So, if the above fix doesn't work for you in that scenario, you may need to look into which dependencies in your project are including xml security and deciding which one you want to include and which one you want to exclude (e.g. using Maven's exclusion or provided scope markings in the given pom).

DRY way of defining dependencies for both jetty:run and deployment

I'm wrapping up the build for a web project which supports two ways of running:
locally using mvn jetty-run;
deployed on an application sever.
For the application server many libraries are marked as provided, since otherwise classpath conflicts occur. At the same time, I have redeclared these dependencies as compile dependencies for the jetty-maven-plugin, since otherwise the goals does not run properly.
The build works like this, but I have a large number of duplicated libraries. Is there a cleaner way of doing this?
Just to follow up on this JETTY-429 has been merged so you can with caution add a configuration parameter <useProvidedScope>true</useProvidedScope>. This will included the provided dependencies on the jetty plugin classpath.
It is worth reading JETTY-429 for details on the potential issues using this option can bring.
Well there's always the wicket solution. (It doesn't have to do anything with wicket, but it can be found in the wicket maven archetype.)
Use a main class that starts jetty programmatically with the project as webapp context. This should pick up all maven dependencies even in provided scope on all major IDEs. Here is such a class:
public class Start{
private static final Logger LOG = Logger.getLogger(Start.class);
public static void main(final String[] args) throws Exception{
LOG.addAppender(new ConsoleAppender(new SimpleLayout(), "system.out"));
final Server server = new Server();
final SocketConnector connector = new SocketConnector();
// Set some timeout options to make debugging easier.
connector.setMaxIdleTime(1000 * 60 * 60);
connector.setSoLingerTime(-1);
connector.setPort(9090);
server.setConnectors(new Connector[] { connector });
final WebAppContext bb = new WebAppContext();
bb.setServer(server);
bb.setContextPath("/");
bb.setWar("src/main/webapp");
server.addHandler(bb);
try{
LOG.info(//
">>> STARTING EMBEDDED JETTY SERVER, PRESS ANY KEY TO STOP" //
);
server.start();
System.in.read();
LOG.info(">>> STOPPING EMBEDDED JETTY SERVER");
server.stop();
server.join();
} catch(final Exception e){
LOG.error("Something bad happened", e);
System.exit(100);
}
}
// CHECKSTYLE:ON
}
The nice part: you can launch this as a standard eclipse run configuration, including easy debugging.
The downside: you need jetty as an additional provided dependency:
<!-- JETTY DEPENDENCIES FOR TESTING -->
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty</artifactId>
<version>${jetty.version}</version>
<scope>provided</scope>
</dependency>
<dependency>
<groupId>org.mortbay.jetty</groupId>
<artifactId>jetty-util</artifactId>
<version>${jetty.version}</version>
<scope>provided</scope>
</dependency>
I'm not saying this is the best solution, but how about doing this:
Your default dependencies list the provided dependencies as "provided". The default project configuration should produce WAR files which are going to run on non-Jetty servers.
Define a profile "jetty" and redeclare the provided dependencies in the compile scope. Then just run mvn -Pjetty jetty:run
It might not be the cleanest solution in the world, as it forces some repetition, but it should get the job done for you.
I'd also like some kind of flag to be able to include provided scoped dependencies when running Jetty. Actually, Jetty already has a useTestClasspath doing something similar for test scoped dependencies (why not including provided dependencies in that case) in order to avoid having to duplicate dependencies in a profile. This is tracked by JETTY-429 which has a patch for such a flag.