I already have gzip turned on for spring boot embedded server. My concern is how spring boot handles the gzip of static resources. Since these do not change, does spring boot (or the underlying embedded server) run the gzip algorithm once then caches the result? It just seems to be a waste of processing power to run the gzip algo on every request for a static resource.
The GzipResourceResolver looks for *.gz files in your resource directory, which you must prepare at build time. It does not gzip the resources on the fly, which addresses your concerns, I think.
The main docs do not directly cover this: https://docs.spring.io/spring-boot/docs/current-SNAPSHOT/reference/htmlsingle/#boot-features-spring-mvc-static-content
You can configure this by
1) At build time, create gzip copies of all your static files, e.g. by:
gzip --keep --best -r src/main/resources/public
2) Configure spring, by adding a class like:
#Configuration
public class MvcConfig extends WebMvcConfigurerAdapter {
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry
.addResourceHandler("/**")
.addResourceLocations("/public/")
.resourceChain(true) // cache resource lookups
.addResolver(new GzipResourceResolver())
.addResolver(new PathResourceResolver());
}
}
You would have to use the Spring ResourceResolver specifically around caching, the CachingResourceResolver. Your configuration should be something along the lines of this:
#Configuration
#EnableWebMvc
#EnableCaching
public class MvcConfig extends WebMvcConfigurerAdapter {
#Autowired
private CacheManager cacheManager;
#Override
public void addResourceHandlers(ResourceHandlerRegistry registry) {
registry
.addResourceHandler("/resources/**")
.addResourceLocations("/resources/")
.setCachePeriod(3600) // Browser cache
.resourceChain(true)
.addResolver(new CachingResourceResolver(cacheManager, "resourceCache"))
.addResolver(new GzipResourceResolver())
.addResolver(new PathResourceResolver());
}
}
The important thing is that the CacheManager has to be configured for the application so check out the Spring Boot Caching docs for more info (probably best for a local cache using Caffeine).
Related
How to customize the reactor access log in Spring webflux?
I am able to turn on reactor netty access log by setting
-Dreactor.netty.http.server.accessLogEnabled=true
I would like to customize the format, eg: I need a few request headers to be logged and remove the IP address.
Any hints to achieve this in Spring Webflux application would be helpful.
You can do it programmatically like this
#Component
public class MyNettyWebServerCustomizer
implements WebServerFactoryCustomizer<NettyReactiveWebServerFactory> {
#Override
public void customize(NettyReactiveWebServerFactory factory) {
factory.addServerCustomizers(httpServer -> httpServer.accessLog(true, x -> AccessLog.create("method={}, uri={}", x.method(), x.uri())));
}
}
More about custom access logging you can find in the documentation
To create end-to-end local tests of data workflow I utilize "mock S3" container (e.g adobe/S3Mock). Seems to work just fine. However, some parts of the system rely on S3A client. As far as I see, its format does not allow to point to particular nameserver or endpoint.
Is it possible to make S3A work in local environment?
you talking about the ASF Hadoop S3A Connector? Nobody has tested against S3 mock AFAIK (never seen it before!), but it does work with non-AWS endpoints
set fs.s3a.endpoint to the URL of your S3 connection. There's some settings about switching from https to http (fs.s3a.connection.ssl.enabled = false) and moving from virtual hosts to directories (fs.s3a.path.style.access = true) which will also be needed.
further reading
Like I said: nobody has done this. We developers just go against the main AWS endpoints with its problems (latency, inconsistency, error reporting, etc), precisely because its what you get in production. But for your local testing, it will simplify your life (and you can run it under jenkins without having to give it any secrets)
Answer by #stevel worked for me. Here is the code if someone wants to refer
class S3WriterTest {
private static S3Mock api;
private static AmazonS3 mockS3client;
#BeforeAll
public static void setUp() {
//start mock s3 service using findify
api = new S3Mock.Builder().withPort(8001).withInMemoryBackend().build();
api.start();
/* AWS S3 client setup.
* withPathStyleAccessEnabled(true) trick is required to overcome S3 default
* DNS-based bucket access scheme
* resulting in attempts to connect to addresses like "bucketname.localhost"
* which requires specific DNS setup.
*/
EndpointConfiguration endpoint = new EndpointConfiguration("http://localhost:8001", "us-west-2");
mockS3client = AmazonS3ClientBuilder
.standard()
.withEndpointConfiguration(endpoint)
.withPathStyleAccessEnabled(true)
.withCredentials(new AWSStaticCredentialsProvider(new AnonymousAWSCredentials()))
.build();
mockS3client.createBucket("test-bucket");
}
#AfterAll
public static void tearDown() {
api.shutdown();
}
#Test
void unitTestForHadoopCodeWritingUsingS3A {
Configuration hadoopConfig = getTestConfiguration();
........
}
private static Configuration getTestConfiguration() {
Configuration config = new Configuration();
config.set("fs.s3a.endpoint", "http://127.0.0.1:8001");
config.set("fs.s3a.connection.ssl.enabled", "false");
config.set("fs.s3a.path.style.access", "true");
config.set("fs.s3a.aws.credentials.provider", "org.apache.hadoop.fs.s3a.SimpleAWSCredentialsProvider");
config.set("fs.s3a.access.key", "foo");
config.set("fs.s3a.secret.key", "bar");
return config;
}
}
I was wondering if it's possible to use #Cacheable annotations on the spring data mongo repositories. For example like this:
public interface UserRepository extends MongoRepository<User, String> {
#Cacheable("byId")
public interface UserRepository extends MongoRepository<User, String> {
User findById(String id);
}
}
I'd like to do it on the interface class itself and avoid having a wrapper class if possible. In addition, is there a sample for how to do the cache config for redis with java config (not xml)?
Yes, you can use Cacheable on any public method with spring aspects. You also have to use EnableCaching in any configuration class and optionally a CacheManager bean.
I am writing integration tests using Arquillian with embedded glassfish 3.1.2.2 using TestNG. I want to be able to run those tests in parallel, and for this case i need to dynamically configure glassfish ports and database name (we already have this set-up, and I want to reuse it of arquillian tests). What I am missing is a 'before container start' hook, where I could prepare the database, lookup free ports and update my glassfish configuration (domain.xml, could also be glassfish-resources.xml). Is there a 'clean' solution for this, or my usecase was not foreseen by Arquillian developers?
The hacky way I solved it currently is to override arquillian's beforeSuite method but this one gets called twice - at test startup and then in the container (therefore my pathetic static flag). Secondly, this solution would not work for JUnit based tests as there's no way to intercept arquillian's before suite:
public class FullContainerIT extends Arquillian {
private static boolean dbInitialized;
//#RunAsClient <-supported by #Test only
#Override
#BeforeSuite(groups = "arquillian", inheritGroups = true)
public void arquillianBeforeSuite() throws Exception {
if (dbInitialized == false) {
initializeDb();
dbInitialized = true;
}
super.arquillianBeforeSuite();
}
}
Some ideas I had:
+ having #BeforeSuite #RunAsClient seems to be what I need, but #RunAsClient is supported for #Test only;
+ I have seen org.jboss.arquillian.container.spi.event.container.BeforeStart event in Arquillian JavaDocs, but I have no clue how to listen to Arquillian events;
+ I have seen there is a possibility to have #Deployment creating a ShrinkWrap Descriptor, but these do not support Glassfish resources.
I found a clean solution for my problem on JBoss forum. You can register a LoadableExtension SPI and modify the arquillian config (loaded from xml). This is where I can create a database and filter glassfish-resources.xml with proper values. The setup looks like this:
package com.example.extenstion;
public class AutoDiscoverInstanceExtension
implements org.jboss.arquillian.core.spi.LoadableExtension {
#Override
public void register(ExtensionBuilder builder) {
builder.observer(LoadContainerConfiguration.class);
}
}
package com.example.extenstion;
public class LoadContainerConfiguration {
public void registerInstance(#Observes ContainerRegistry, ServiceLoader serviceLoader) {
//Do the necessary setup here
String filteredFilename = doTheFiltering();
//Get the container defined in arquillian.xml and modify it
//"default" is the container's qualifier
Container definition = registry.getContainer("default");
definition.getContainerConfiguration()
.property("resourcesXml", filteredFilename);
}
}
You also need to configure the SPI Extension by creating a file
META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension
with this contents:
com.example.extenstion.AutoDiscoverInstanceExtension
I want to put all my configuration files (arquillian.xml, glassfish-resources.xml,test-persistence.xml) in a library and just reference that when i want to run a test in a project. But it seems the resourceXml only allows for a file-path and not a classpath: variable.
Is there a way to do that?
My configuration is currently based upon the arquillian-persistence-tutorial example project. I actually have no special configuration which is need. Just a in-memory db etc. etc.
I fixed it by extending the Arquillian class, and writing the resource to a local file (in the target folder) before any test.
Now in a domain project i only need to do this in a unit test:
public class MyEntityRepositoryImplTest extends BaseArquillianTest {
#Deployment
public static Archive<?> createDeployment() {
return create().//
addClass(MyEntityRepositoryImpl.class).//
addClass(MyEntityDaoImpl.class).//
addClass(MockSomeRepositoryImpl.class).//
addClass(MyEntityJpa.class);
}
#EJB
private MyEntityRepository myEntityRepository;
#Test
public void mMytest () {
}