How to make versioned KV store work with VaultPropertySource - spring-vault

I am trying to make versioned KV store of vault work with VaultPropertySource so that property can be accessed using #Value. However it is not working as expected. I am using 2.1.2.RELEASE version of spring-vault-core. The intention is to make it work with spring vault and Spring MVC.
I have already tried with #import(EnvironmentVaultConfiguration.class) to no avail.
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.PropertySource;
import org.springframework.core.env.ConfigurableEnvironment;
import org.springframework.core.env.MutablePropertySources;
import org.springframework.vault.authentication.ClientAuthentication;
import org.springframework.vault.authentication.TokenAuthentication;
import org.springframework.vault.client.VaultEndpoint;
import org.springframework.vault.config.AbstractVaultConfiguration;
import org.springframework.vault.core.VaultTemplate;
import org.springframework.vault.core.env.VaultPropertySource;
import javax.annotation.PostConstruct;
import java.net.URI;
import java.util.List;
#Configuration
#PropertySource("vault.properties")
public class AppConfig extends AbstractVaultConfiguration {
#Value("${vault.uri}")
private URI vaultUri;
#Value("${vault.token}")
private String token;
#Value("#{'${vault.sources:}'.split(',')}")
private List<String> vaultSources;
#Autowired
private ConfigurableEnvironment environment;
#Autowired
private VaultTemplate vaultTemplate;
/**
* Specify an endpoint for connecting to Vault.
*/
#Override
public VaultEndpoint vaultEndpoint() {
return VaultEndpoint.from(vaultUri);
}
/**
* Configure a client authentication.
* Please consider a more secure authentication method
* for production use.
*/
#Override
public ClientAuthentication clientAuthentication() {
return new TokenAuthentication(token);
}
#PostConstruct
public void setPropertySource() {
MutablePropertySources sources = environment.getPropertySources();
vaultSources.stream().forEach(vs -> {
sources.addFirst(new VaultPropertySource(vaultTemplate, vs));
});
}
}
In the given code, if I provide
vault.sources=secret/data/abcd,secret/data/pqrs
then it works and returns secrets with data. and metadata. prefix. Which means that it is using generic approach to fetch secrets and not kv one.
If I remove data from path i.e. vault.sources=secret/abcd,secret/pqrs, it simply does not connect and throws exception with 403. This means that it must not be using kv v2.
Can someone please help me with how to use Versioned API of spring-vault in this code?

Key-Value 2 support using VaultPropertySource is not yet released. It will be shipped with Spring Vault 2.2 (see this GitHub issue).
Until then, you can use snapshot builds to verify the code is helpful for your use case.

Based on Mark's reponse above, I decided to use VaultPropertySource with PropertyTransformer until we get KV version2 support out of the box.
public class DataMetadataPrefixRemoverPropertyTransformer implements PropertyTransformer {
private final String dataPrefix = "data.";
private final String metadataPrefix = "metadata.";
public Map<String, Object> transformProperties(Map<String, ? extends Object> inputProperties) {
Map<String, Object> target = new LinkedHashMap(inputProperties.size(), 1.0F);
Iterator propertiesIterator = inputProperties.entrySet().iterator();
while(propertiesIterator.hasNext()) {
Map.Entry<String, ? extends Object> entry = (Map.Entry)propertiesIterator.next();
String key = entry.getKey();
// do not add metadata properties to environment for now - do not see a use case for it as of now.
if (StringUtils.startsWithIgnoreCase(key, metadataPrefix)) {
continue;
}
if (StringUtils.startsWithIgnoreCase(key, dataPrefix)) {
key = StringUtils.replace(key, dataPrefix, "");
}
target.put(key, entry.getValue());
}
return target;
}
}
Hope it can help someone looking for similar solution.

Related

Spring Boot - Tables are not being created automatically

I am unable to get spring to create tables automatically. I tried every solution I could find regarding the properties and annotation that should be added.
Spring version - 5.2.5.RELEASE
My application properties looks like that:
spring.datasource.url=jdbc:mysql://localhost:3306/ci_test?useLegacyDatetimeCode=false&serverTimezone=UTC
spring.datasource.username=****
spring.datasource.password=****
spring.datasource.driver-class-name=com.mysql.cj.jdbc.Driver
spring.jpa.database-platform=org.hibernate.dialect.SQLServerDialect
spring.jpa.generate-ddl=true
spring.jpa.hibernate.ddl-auto=update
management.endpoints.web.exposure.include=*
management.endpoint.shutdown.enabled=true
#management.server.port=8090
logging.level.org.springframework=INFO
My Main app:
com.mypackage.autoTester;
import org.apache.logging.log4j.LogManager;
import org.apache.logging.log4j.Logger;
import org.modelmapper.ModelMapper;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.domain.EntityScan;
import org.springframework.boot.context.properties.EnableConfigurationProperties;
import org.springframework.context.annotation.Bean;
import org.springframework.core.SpringVersion;
import org.springframework.scheduling.annotation.EnableScheduling;
#SpringBootApplication
#EnableScheduling
#EnableConfigurationProperties
#EntityScan(basePackages = {"com.mypackage.autoTester"}) // scan JPA entities
public class AutoTesterApplication {
public static final Logger logger = LogManager.getLogger();
#Bean
public ModelMapper modelMapper() {
return new ModelMapper();
}
public static void main(String[] args) {
SpringApplication.run(AutoTesterApplication.class, args);
}
}
My Entity:
com.mypackage.autoTester.entities;
import lombok.*;
import javax.persistence.Entity;
import javax.persistence.Id;
import javax.persistence.Table;
#Entity
#Table(name = "testTable")
#Getter
#Setter
#AllArgsConstructor(access = AccessLevel.PUBLIC)
#NoArgsConstructor
public class TestEntity {
#Id
private long id;
}
Is there something wrong with the code above? Am I missing something?
UPDATE:
I tried the solutions below but no help :(
Back in the days I used spring boot plugin in eclipse and it worked, I now work with intellij so I am not sure if it is related
You need to update the ddl-auto to create instead of update in your application.properties
spring.jpa.hibernate.ddl-auto=create
Some of the values it accepts are:
create: creates the schema.
update: updates the schema.
validate: validates the schema.
create-drop: creates the schema and destroys the data previously present.
But this is not recommended for production databases, you can use flyway if you want incremental changes to be made to your db instead of hibernate update.
try to add to your application.properties :
spring.jpa.hibernate.ddl-auto=update
spring.jpa.properties.hibernate.dialect=org.hibernate.dialect.MySQL5InnoDBDialect
I found the solution and perhaps my question was lacking some information so it seems.
We are using 2 DBs and we initialize them manually in our code with the help of #EnableJpaRepositories
#Configuration
#EnableJpaRepositories(basePackages = "",
entityManagerFactoryRef = "primaryEntityManager",
transactionManagerRef = "primaryTransactionManager")
public class PrimaryConfig {
private final Environment env;
public PrimaryConfig(Environment env) {
this.env = env;
}
#Bean
#ConfigurationProperties(prefix="spring.datasource")
public DataSource primaryDataSource() {
DriverManagerDataSource dataSource = new DriverManagerDataSource();
dataSource.setUrl(env.getProperty("spring.datasource.url"));
dataSource.setUsername(env.getProperty("spring.datasource.username"));
dataSource.setPassword(env.getProperty("spring.datasource.password"));
dataSource.setDriverClassName(Objects.requireNonNull(env.getProperty("spring.datasource.driver-class-name")));
return dataSource;
}
#Bean
public PlatformTransactionManager primaryTransactionManager() {
JpaTransactionManager transactionManager
= new JpaTransactionManager();
transactionManager.setEntityManagerFactory(
primaryEntityManager().getObject());
return transactionManager;
}
#Bean
public LocalContainerEntityManagerFactoryBean primaryEntityManager() {
final LocalContainerEntityManagerFactoryBean em = new LocalContainerEntityManagerFactoryBean();
em.setDataSource(primaryDataSource());
em.setPackagesToScan("");
final HibernateJpaVendorAdapter vendorAdapter = new HibernateJpaVendorAdapter();
vendorAdapter.setGenerateDdl(true); // -> added this line
em.setJpaVendorAdapter(vendorAdapter);
return em;
}
}
So I needed to add only one line for it to work and start executing the SQL commands automatically.
It appears that since we created it manually, the properties added to the application.properties file didn't make any difference, or at least some of them.
I hope it will help other people who see this also since I wasted a lot of time on this :)

Arquillian Graphene #Location placeholder

I'm learning Arquillian right now I wonder how to create page that has a placeholder inside the path. For example:
#Location("/posts/{id}")
public class BlogPostPage {
public String getContent() {
// ...
}
}
or
#Location("/posts/{name}")
#Location("/specific-page?requiredParam={value}")
I have looking for an answer on graphine and arquillian reference guides without success. I used library from other language that have support for page-objects, but it has build-in support for placeholders.
AFAIK there is nothing like this implemented in Graphene.
To be honest, I'm not sure how this should behave - how would you pass the values...?
Apart from that, I think that it could be also limited by Java annotation abilities https://stackoverflow.com/a/10636320/6835063
This is not possible currently in Graphene. I've created ARQGRA-500.
It's possible to extend Graphene to add dynamic parameters now. Here's how. (Arquillian 1.1.10.Final, Graphene 2.1.0.Final.)
Create an interface.
import java.util.Map;
public interface LocationParameterProvider {
Map<String, String> provideLocationParameters();
}
Create a custom LocationDecider to replace the corresponding Graphene's one. I replace the HTTP one. This Decider will add location parameters to the URI, if it sees that the test object implements our interface.
import java.io.UnsupportedEncodingException;
import java.net.URLEncoder;
import java.util.Map;
import java.util.Map.Entry;
import org.jboss.arquillian.core.api.Instance;
import org.jboss.arquillian.core.api.annotation.Inject;
import org.jboss.arquillian.graphene.location.decider.HTTPLocationDecider;
import org.jboss.arquillian.graphene.spi.location.Scheme;
import org.jboss.arquillian.test.spi.context.TestContext;
public class HTTPParameterizedLocationDecider extends HTTPLocationDecider {
#Inject
private Instance<TestContext> testContext;
#Override
public Scheme canDecide() {
return new Scheme.HTTP();
}
#Override
public String decide(String location) {
String uri = super.decide(location);
// not sure, how reliable this method of getting the current test object is
// if it breaks, there is always a possibility of observing
// org.jboss.arquillian.test.spi.event.suite.TestLifecycleEvent's (or rather its
// descendants) and storing the test object in a ThreadLocal
Object testObject = testContext.get().getActiveId();
if (testObject instanceof LocationParameterProvider) {
Map<String, String> locationParameters =
((LocationParameterProvider) testObject).provideLocationParameters();
StringBuilder uriParams = new StringBuilder(64);
boolean first = true;
for (Entry<String, String> param : locationParameters.entrySet()) {
uriParams.append(first ? '?' : '&');
first = false;
try {
uriParams.append(URLEncoder.encode(param.getKey(), "UTF-8"));
uriParams.append('=');
uriParams.append(URLEncoder.encode(param.getValue(), "UTF-8"));
} catch (UnsupportedEncodingException e) {
throw new RuntimeException(e);
}
}
uri += uriParams.toString();
}
return uri;
}
}
Our LocationDecider must be registered to override the Graphene's one.
import org.jboss.arquillian.core.spi.LoadableExtension;
import org.jboss.arquillian.graphene.location.decider.HTTPLocationDecider;
import org.jboss.arquillian.graphene.spi.location.LocationDecider;
public class MyArquillianExtension implements LoadableExtension {
#Override
public void register(ExtensionBuilder builder) {
builder.override(LocationDecider.class, HTTPLocationDecider.class,
HTTPParameterizedLocationDecider.class);
}
}
MyArquillianExtension should be registered via SPI, so create a necessary file in your test resources, e.g. for me the file path is src/test/resources/META-INF/services/org.jboss.arquillian.core.spi.LoadableExtension. The file must contain a fully qualified class name of MyArquillianExtension.
And that's it. Now you can provide location parameters in a test.
import java.util.HashMap;
import java.util.Map;
import org.jboss.arquillian.graphene.page.InitialPage;
import org.jboss.arquillian.graphene.page.Location;
import org.junit.Test;
public class TestyTest implements LocationParameterProvider {
#Override
public Map<String, String> provideLocationParameters() {
Map<String, String> params = new HashMap<>();
params.put("mykey", "myvalue");
return params;
}
#Test
public void test(#InitialPage TestPage page) {
}
#Location("MyTestView.xhtml")
public static class TestPage {
}
}
I've focused on parameters specifically, but hopefully this paves the way for other dynamic path manipulations.
Of course this doesn't fix the Graphene.goTo API. This means before using goTo you have to provide parameters via this roundabout provideLocationParameters way. It's weird. You can make your own alternative API, goTo that accepts parameters, and modify your LocationDecider to support other ParameterProviders.

Customize login in Grails Spring Security plugin

I have an application where the login should include an organization number, so the login needs to be username + password + organization number.
Sample case: If the username + password matches with an existing user, I need to check if that user has the organization id. If not, the login should fail.
I saw that the login form from spring security plugin submits to /app/j_spring_security_check but couldn't find where that is actually implemented.
Also I'm not sure if touching that is the right way of implementing this custom login.
My question is where / how to customize the login action? (to make it fail on the case I described above).
We can do this by overriding the filter UserNamePasswordAuthenticationFilter and provide our custom attemptAuthentication.
So, go to DefaultSecurityConfig.groovy file (inside plugins). See tree diagram below:
target
|-work
|-plugins
|-spring-security-core-2.0-RC5
|-conf
|-DefaultSecurityConfig.groovy
In DefaultSecurityConfig.groovy under apf closure we specify filterProcessUrl which we can override in grails application's Config.groovy like we do for other properties (e.g. rejectIfNoRule)
grails.plugin.springsecurity.apf.filterProcessesUrl="your url"
Now we understood how it checks for authentication.Let's customise it own way by overriding the method attemptAuthentication of filter named UsernamePasswordAuthenticationFilter. For example, see below(also, go through the inline comments added there)
package org.springframework.security.web.authentication;
import javax.servlet.http.HttpServletRequest;
import javax.servlet.http.HttpServletResponse;
import org.springframework.security.authentication.AuthenticationServiceException;
import org.springframework.security.authentication.UsernamePasswordAuthenticationToken;
import org.springframework.security.core.Authentication;
import org.springframework.security.core.AuthenticationException;
import org.springframework.security.web.authentication.AbstractAuthenticationProcessingFilter;
import org.springframework.util.Assert;
public class CustomUsernamePasswordAuthenticationFilter extends AbstractAuthenticationProcessingFilter {
public static final String SPRING_SECURITY_FORM_USERNAME_KEY = "j_username";
public static final String SPRING_SECURITY_FORM_PASSWORD_KEY = "j_password";
/** #deprecated */
#Deprecated
public static final String SPRING_SECURITY_LAST_USERNAME_KEY = "SPRING_SECURITY_LAST_USERNAME";
private String usernameParameter = "j_username";
private String passwordParameter = "j_password";
private String organisationParameter = 'j_organisation'
private boolean postOnly = true;
public UsernamePasswordAuthenticationFilter() {
super("/j_spring_security_check");
}
public Authentication attemptAuthentication(HttpServletRequest request, HttpServletResponse response) throws AuthenticationException {
if(this.postOnly && !request.getMethod().equals("POST")) {
throw new AuthenticationServiceException("Authentication method not supported: " + request.getMethod());
} else {
String username = this.obtainUsername(request);
String password = this.obtainPassword(request);
String password = this.obtainOrganisation(request);
//regular implementation in spring security plugin /**
UsernamePasswordAuthenticationToken authRequest = new UsernamePasswordAuthenticationToken(username, password);
this.setDetails(request, authRequest);
return this.getAuthenticationManager().authenticate(authRequest);
}
**/
//Your custom implementation goes here(Authenticate on the basis of organisation as well). Here you need to customise authenticate as per your requirement so that it checks for organisation as well.
}
protected String obtainOrganisation(HttpServletRequest request) {
return request.getParameter(this.organisationParameter);
}
protected String obtainPassword(HttpServletRequest request) {
return request.getParameter(this.passwordParameter);
}
protected String obtainUsername(HttpServletRequest request) {
return request.getParameter(this.usernameParameter);
}
protected void setDetails(HttpServletRequest request, UsernamePasswordAuthenticationToken authRequest) {
authRequest.setDetails(this.authenticationDetailsSource.buildDetails(request));
}
public void setUsernameParameter(String usernameParameter) {
Assert.hasText(usernameParameter, "Username parameter must not be empty or null");
this.usernameParameter = usernameParameter;
}
public void setPasswordParameter(String passwordParameter) {
Assert.hasText(passwordParameter, "Password parameter must not be empty or null");
this.passwordParameter = passwordParameter;
}
public void setPostOnly(boolean postOnly) {
this.postOnly = postOnly;
}
public final String getUsernameParameter() {
return this.usernameParameter;
}
public final String getPasswordParameter() {
return this.passwordParameter;
}
}
Hence, it's more of a overriding task in terms of spring security.
To get more clearer idea about same read this nice link for java
and
for grails read this
Hope it helps.
These blogs gives a more detailed idea of the same requirements.

JUnit reporter does not show detailed report for each step in JBehave

I'm trying to set up JBehave for testing web services.
Template story is running well, but I can see in JUnit Panel only Acceptance suite class execution result. What I want is to see execution result for each story in suite and for each step in story like it is shown in simple JUnit tests or in Thucydides framework.
Here is my acceptance suite class: so maybe I Haven't configured something, or either I have to notate my step methods some other way, but I didn't find an answer yet.
package ***.qa_webservices_testing.jbehave;
import java.util.Arrays;
import java.util.List;
import java.util.Properties;
import org.jbehave.core.Embeddable;
import org.jbehave.core.configuration.Configuration;
import org.jbehave.core.configuration.MostUsefulConfiguration;
import org.jbehave.core.io.CodeLocations;
import org.jbehave.core.io.LoadFromClasspath;
import org.jbehave.core.io.StoryFinder;
import org.jbehave.core.junit.JUnitStories;
import org.jbehave.core.parsers.RegexPrefixCapturingPatternParser;
import org.jbehave.core.reporters.CrossReference;
import org.jbehave.core.reporters.Format;
import org.jbehave.core.reporters.StoryReporterBuilder;
import org.jbehave.core.steps.InjectableStepsFactory;
import org.jbehave.core.steps.InstanceStepsFactory;
import org.jbehave.core.steps.ParameterConverters;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import ***.qa_webservices_testing.jbehave.steps.actions.TestAction;
/**
* suite class.
*/
public class AcceptanceTestSuite extends JUnitStories {
private static final String CTC_STORIES_PATTERN = "ctc.stories";
private static final String STORY_BASE = "src/test/resources";
private static final String DEFAULT_STORY_NAME = "stories/**/*.story";
private static final Logger LOGGER = LoggerFactory.getLogger(AcceptanceTestSuite.class);
private final CrossReference xref = new CrossReference();
public AcceptanceTestSuite() {
configuredEmbedder()
.embedderControls()
.doGenerateViewAfterStories(true)
.doIgnoreFailureInStories(false)
.doIgnoreFailureInView(true)
.doVerboseFailures(true)
.useThreads(2)
.useStoryTimeoutInSecs(60);
}
#Override
public Configuration configuration() {
Class<? extends Embeddable> embeddableClass = this.getClass();
Properties viewResources = new Properties();
viewResources.put("decorateNonHtml", "true");
viewResources.put("reports", "ftl/jbehave-reports-with-totals.ftl");
// Start from default ParameterConverters instance
ParameterConverters parameterConverters = new ParameterConverters();
return new MostUsefulConfiguration()
.useStoryLoader(new LoadFromClasspath(embeddableClass))
.useStoryReporterBuilder(new StoryReporterBuilder()
.withCodeLocation(CodeLocations.codeLocationFromClass(embeddableClass))
.withDefaultFormats()
.withViewResources(viewResources)
.withFormats(Format.CONSOLE, Format.TXT, Format.HTML_TEMPLATE, Format.XML_TEMPLATE)
.withFailureTrace(true)
.withFailureTraceCompression(false)
.withMultiThreading(false)
.withCrossReference(xref))
.useParameterConverters(parameterConverters)
// use '%' instead of '$' to identify parameters
.useStepPatternParser(new RegexPrefixCapturingPatternParser(
"%"))
.useStepMonitor(xref.getStepMonitor());
}
#Override
protected List<String> storyPaths() {
String storiesPattern = System.getProperty(CTC_STORIES_PATTERN);
if (storiesPattern == null) {
storiesPattern = DEFAULT_STORY_NAME;
} else {
storiesPattern = "**/" + storiesPattern;
}
LOGGER.info("will search stories by pattern {}", storiesPattern);
List<String> result = new StoryFinder().findPaths(STORY_BASE, Arrays.asList(storiesPattern), Arrays.asList(""));
for (String item : result) {
LOGGER.info("story to be used: {}", item);
}
return result;
}
#Override
public InjectableStepsFactory stepsFactory() {
return new InstanceStepsFactory(configuration(), new TestAction());
}
}
my test methods look like:
Customer customer = new cutomer();
#Given ("I have Access to Server")
public void givenIHaveAccesToServer() {
customer.haveAccesToServer();
}
So they are notated only by JBehave notations.
The result returned in Junit panel is only like here (I yet have no rights to post images):
You should try this open source library:
https://github.com/codecentric/jbehave-junit-runner
It does exactly what you ask for :)
Yes, the codecentric runner works very nicely.
https://github.com/codecentric/jbehave-junit-runner

Play 2 framework testing simulate session and POST?

When running a web test like this
#Test
public void runInBrowser() {
running(testServer(3333), HtmlUnitDriver.class, new Callback<TestBrowser>() {
public void invoke(TestBrowser browser) {
browser.goTo("http://localhost:3333");
assertThat(browser.$("#title").getTexts().get(0)).isEqualTo("Hello Guest");
browser.$("a").click();
assertThat(browser.url()).isEqualTo("http://localhost:3333/Coco");
assertThat(browser.$("#title", 0).getText()).isEqualTo("Hello Coco");
}
});
}
How can one pass sessions values while using this kind of testing and how can one simulate a POST? Thanks :-)
These are Selenium tests with FluentLenium. Since you test with a browser you must use an HTML form with method POST to make a POST request.
browser.goTo("http://localhost:3333" + routes.Login.login().url());//example for reverse route, alternatively use something like "http://localhost:3333/login"
browser.fill("#password").with("secret");
browser.fill("#username").with("aUsername");
browser.submit("#signin");//trigger submit button on the form
//after finished request: http://www.playframework.org/documentation/api/2.0.4/java/play/test/TestBrowser.html
browser.getCookies(); //read only cookies
Maybe you don't want to make test with a browser but instead with HTTP you can use FakeRequests:
import static controllers.routes.ref.Application;
import static org.fest.assertions.Assertions.assertThat;
import static play.mvc.Http.Status.OK;
import static play.mvc.Http.Status.UNAUTHORIZED;
import static play.test.Helpers.*;
import play.libs.WS;
import java.util.HashMap;
import java.util.Map;
import org.junit.BeforeClass;
import org.junit.Test;
import play.mvc.Result;
import play.test.FakeRequest;
public class SoTest {
#Test
public void testInServer() {
running(testServer(3333), new Runnable() {
public void run() {
Fixtures.loadAll();//you may have to fill your database you have to program this yourself
Map<String, String> parameters = new HashMap<String, String>();
parameters.put("userName", "aUsername");
parameters.put("password", "secret");
FakeRequest fakeRequest = new FakeRequest().withSession("key", "value").withCookies(name, value, maxAge, path, domain, secure, httpOnly).withFormUrlEncodedBody(parameters);
Result result = callAction(Application.signIn(), fakeRequest);
int responseCode = status(result);
assertThat(responseCode).isEqualTo(OK);
}
});
}
}
Also check out this answer: How to manipulate Session, Request and Response for test in play2.0