FlatFileParseException Parsing class path resource error in spring batch - kotlin

I want to set classPath resource of csv file . Which is present in my
project resource folder. When set the class Path and run the project I
get issue in runtime issue of classpath resource
here is the issue
org.springframework.batch.item.file.FlatFileParseException: Parsing error at line: 1 in resource=[class path resource [result-match-metadata.csv]], input=[id city date player_of_match venue neutral_venue team1 team2 toss_winner toss_decision winner result result_margin eliminator method umpire1 umpire2]
result-match-metadata.csv file is present inside my resource folder
Batchconfig.kt
package com.nilmani.dashboardipl.data
import com.nilmani.dashboardipl.entity.Match
import org.springframework.batch.core.Job
import org.springframework.batch.core.Step
import org.springframework.batch.core.configuration.annotation.EnableBatchProcessing
import org.springframework.batch.core.configuration.annotation.JobBuilderFactory
import org.springframework.batch.core.configuration.annotation.StepBuilderFactory
import org.springframework.batch.core.launch.support.RunIdIncrementer
import org.springframework.batch.item.database.BeanPropertyItemSqlParameterSourceProvider
import org.springframework.batch.item.database.JdbcBatchItemWriter
import org.springframework.batch.item.database.builder.JdbcBatchItemWriterBuilder
import org.springframework.batch.item.file.FlatFileItemReader
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder
import org.springframework.batch.item.file.mapping.BeanWrapperFieldSetMapper
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.context.annotation.Bean
import org.springframework.context.annotation.Configuration
import org.springframework.core.io.ClassPathResource
import javax.sql.DataSource
#Configuration
#EnableBatchProcessing
class BatchConfig {
#Autowired
private lateinit var jobBuilderFactory: JobBuilderFactory
#Autowired
private lateinit var stepBuilderFactory: StepBuilderFactory
val FIELD_NAMES = arrayOf(
"id","city","date","player_of_match","venue","neutral_venue",
"team1","team2","toss_winner","toss_decision",
"winner","result","result_margin","eliminator","method","umpire1","umpire2"
)
#Bean
fun reader(): FlatFileItemReader<MatchInput> {
return FlatFileItemReaderBuilder<MatchInput>()
.name("MatchItemReader")
.resource(ClassPathResource("result-match-metadata.csv"))
.delimited()
.names(*FIELD_NAMES)
.fieldSetMapper(object : BeanWrapperFieldSetMapper<MatchInput>() {
init {
setTargetType(MatchInput::class.java)
}
})
.build()
}
}
how to set csv file path in batch config

The resource is correctly found. The error means that the first line cannot be mapped to an instance of MatchInput. You just need to skip the header with FlatFileItemReaderBuilder#linesToSkip(1) in your item reader definition.

Related

How to start a #Bean with custom parameters after an event had happened with spring for tests?

I am working on adding RepositoryTests with TestContainers framework for a project that uses R2dbc and I am running into the following situation:
1 - On the main project I set r2dbc url (with port and hostname) on application.yaml file and spring data manages everything and things just work.
2 - On the Tests however, I am using TestContainers framework more specifically DockerComposeContainer which I use to create a mocked container using docker-compose.test.yaml file with the databases I need.
3 - This container creates a port number on the go I define a port number on my docker-compose file but the port number that DockerComposeContainer will provide me is random and changes everytime I run the tests, what makes having a static url on application-test.yaml not an option anymore.
So I need to dinamically create this bean R2dbcEntityTemplate at run time and only after the DockerComposeContainer will give me the port number. So my application can connect to the correct port and things should work as expected.
I tried to create this class:
package com.wayfair.samworkgroupsservice.adapter
import io.r2dbc.mssql.MssqlConnectionConfiguration
import io.r2dbc.mssql.MssqlConnectionFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.beans.factory.config.ConstructorArgumentValues
import org.springframework.beans.factory.support.BeanDefinitionRegistry
import org.springframework.beans.factory.support.GenericBeanDefinition
import org.springframework.context.ApplicationContext
import org.springframework.context.annotation.Profile
import org.springframework.data.r2dbc.core.DefaultReactiveDataAccessStrategy
import org.springframework.data.r2dbc.core.R2dbcEntityTemplate
import org.springframework.data.r2dbc.dialect.SqlServerDialect
import org.springframework.r2dbc.core.DatabaseClient
import org.springframework.stereotype.Component
#Component
#Profile("test")
class TemplateFactory(
#Autowired val applicationContext: ApplicationContext
) {
private val beanFactory = applicationContext.autowireCapableBeanFactory as BeanDefinitionRegistry
fun registerTemplateBean(host: String, port: Int) {
val beanDefinition = GenericBeanDefinition()
beanDefinition.beanClass = R2dbcEntityTemplate::class.java
val args = ConstructorArgumentValues()
args.addIndexedArgumentValue(
0,
DatabaseClient.builder()
.connectionFactory(connectionFactory(host, port))
.bindMarkers(SqlServerDialect.INSTANCE.bindMarkersFactory)
.build()
)
args.addIndexedArgumentValue(1, DefaultReactiveDataAccessStrategy(SqlServerDialect.INSTANCE))
beanDefinition.constructorArgumentValues = args
beanFactory.registerBeanDefinition("R2dbcEntityTemplate", beanDefinition)
}
// fun entityTemplate(host: String = "localhost", port: Int = 1435) =
// R2dbcEntityTemplate(
// DatabaseClient.builder()
// .connectionFactory(connectionFactory(host, port))
// .bindMarkers(SqlServerDialect.INSTANCE.bindMarkersFactory)
// .build(),
// DefaultReactiveDataAccessStrategy(SqlServerDialect.INSTANCE)
// )
private fun connectionFactory(host: String, port: Int) =
MssqlConnectionFactory(
MssqlConnectionConfiguration.builder()
.host(host)
.port(port)
.username("sa")
.password("Password123##?")
.build()
)
}
And this is how my db initiliser looks like:
package com.wayfair.samworkgroupsservice.adapter.note
import com.wayfair.samworkgroupsservice.adapter.DBInitializerInterface
import com.wayfair.samworkgroupsservice.adapter.TemplateFactory
import org.springframework.beans.factory.annotation.Autowired
import org.springframework.data.r2dbc.core.R2dbcEntityTemplate
import org.testcontainers.containers.DockerComposeContainer
import org.testcontainers.containers.wait.strategy.Wait
import org.testcontainers.junit.jupiter.Container
import org.testcontainers.junit.jupiter.Testcontainers
import java.io.File
#Testcontainers
class NoteTagDBInitializer : DBInitializerInterface {
#Autowired
override lateinit var client: R2dbcEntityTemplate
#Autowired
lateinit var factory: TemplateFactory
override val sqlScripts = listOf(
"db/note/schema.sql",
"db/note/reset.sql",
"db/note/data.sql"
)
init {
factory.registerTemplateBean(
cont.getServiceHost("test-db-local_1", 1433),
cont.getServicePort("test-db-local_1", 1433)
)
}
companion object {
#Container
val cont: KDockerComposerContainer = KDockerComposerContainer("docker-compose.test.yml")
.withExposedService(
"test-db-local_1", 1433,
Wait.forListeningPort()
)
.withLocalCompose(true)
.also {
it.start()
val porttt = it.getServicePort("test-db-local_1", 1433)
print(porttt)
}
class KDockerComposerContainer(yamlFile: String) :
DockerComposeContainer<KDockerComposerContainer>(File(yamlFile))
}
}
I am not getting errors when trying to start this template factory with no useful error message,
But to be honest I don't know anymore if am putting effort into the correct solution, does anyone have any insight on how to pull this off or if I am doing anything wrong here?
So to summarise for production app it is fine, it starts based off of the url on application.yaml file and that's it, but for tests I need something dinamic with ports that will change everytime.
Thank you in advance ))
Spring already has a solution for your problem.
If you're using a quite recent Spring version (>= 5.2.5), you should utilize #DynamicPropertySource in order to adjust your test configuration properties with a dynamic value of the container database port. Read official spring documentation for more details and kotlin code examples.
If you're stuck with an older Spring version, the interface you need is ApplicationContextInitializer. See this spring github issue for a small example.

xd.lck lock file is not removed after store is closed

My assumption on xodus database locking was that closing the entity store would close the database.
I implemented this with a simple example using the use pattern that calls close:
package whatever
import jetbrains.exodus.entitystore.Entity
import kotlinx.dnq.XdEntity
import kotlinx.dnq.XdModel
import kotlinx.dnq.XdNaturalEntityType
import kotlinx.dnq.store.container.StaticStoreContainer
import kotlinx.dnq.util.initMetaData
import kotlinx.dnq.xdRequiredStringProp
import org.junit.Test
import java.nio.file.Files
class UnclosedTest {
private val dbFolder = Files.createTempDirectory(null).toFile()
private val store = StaticStoreContainer.init(
dbFolder = dbFolder,
environmentName = "store"
).also {
XdModel.registerNodes(
Bogus
)
initMetaData(XdModel.hierarchy, it)
}
#Test
fun `lock file is removed when store is closed`() {
store.use { store ->
store.transactional {
Bogus.new {
text = "gnarf"
}
}
}
assert(dbFolder.exists())
assert(dbFolder.isDirectory)
assert(!dbFolder.resolve("xd.lck").exists())
}
class Bogus(entity: Entity) : XdEntity(entity) {
companion object : XdNaturalEntityType<Bogus>()
var text by xdRequiredStringProp()
}
}
Surprisingly, this test fails with the xd.lck file still being present.
How do I close all resources, making sure the lockfile is removed?
The xd.lck file is being released on closing the database, not removed, regardless of which API do you use: Environments, EntityStores, or Xodus-DNQ DSL. See how it is implemented.

How to test Apache HttpClient RequestConfig values are set correctly? No public getters present

I have this class to configure a HttpClient instance:
package com.company.fraud.preauth.service.feignaccertifyclient;
import com.company.fraud.preauth.config.ProviderClientConfig;
import lombok.RequiredArgsConstructor;
import lombok.extern.slf4j.Slf4j;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.apache.http.conn.ssl.TrustSelfSignedStrategy;
import org.apache.http.impl.client.HttpClientBuilder;
import org.apache.http.ssl.SSLContextBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import java.security.KeyManagementException;
import java.security.KeyStoreException;
import java.security.NoSuchAlgorithmException;
#Slf4j
#Configuration
#RequiredArgsConstructor
public class FeignClientConfig {
private final ProviderClientConfig providerClientConfig;
public HttpClient buildHttpClient() throws NoSuchAlgorithmException, KeyStoreException, KeyManagementException {
RequestConfig.Builder requestBuilder = RequestConfig.custom();
requestBuilder.setConnectTimeout(providerClientConfig.getConnectionTimeout());
requestBuilder.setConnectionRequestTimeout(providerClientConfig.getConnectionRequestTimeout());
requestBuilder.setSocketTimeout(providerClientConfig.getSocketTimeout());
SSLContextBuilder builder = new SSLContextBuilder();
builder.loadTrustMaterial(null, new TrustSelfSignedStrategy());
return HttpClientBuilder.create()
.setMaxConnPerRoute(providerClientConfig.getMaxConnectionNumber())
.setDefaultRequestConfig(requestBuilder.build())
.setSSLContext(builder.loadTrustMaterial(null, new TrustSelfSignedStrategy()).build())
.build();
}
}
How to unit test this class, to see into the resulted HttpClient that these values are correctly set?
From the httpClient I cannot get access to its RequestConfig.
I am aware of these two posts:
How do I test a private function or a class that has private methods, fields or inner classes?
(the number of upvotes in this question shows that it is a concurrent and controversial topic in testing, and my situation may offer an example that why we should look into the inner state of an instance in testing, despite that it is private)
Unit test timeouts in Apache HttpClient
(it shows a way of adding an interceptor in code to check configure values, but I don't like it because I want to separate tests with functional codes)
Is there any way? I understand that this class should be tested, right? You cannot blindly trust it to work; and checking it "notNull" seems fragile to me.
This link may point me to the right direction:
https://dzone.com/articles/testing-objects-internal-state
It uses PowerMock.Whitebox to check internal state of an instance.
So I have checked into PowerMock.Whitebox source code, and it turns out reflection is used internally. And, as PowerMock is said to be not compatible with JUnit 5 yet(till now), and I don't want to add another dependency just for testing, so I will test with reflection.
package com.company.fraud.preauth.service.feignaccertifyclient;
import com.company.fraud.preauth.config.PreAuthConfiguration;
import com.company.fraud.preauth.config.ProviderClientConfig;
import com.company.fraud.preauth.config.StopConfiguration;
import org.apache.http.client.HttpClient;
import org.apache.http.client.config.RequestConfig;
import org.junit.jupiter.api.DisplayName;
import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.mockito.Mock;
import org.springframework.boot.test.context.SpringBootTest;
import org.springframework.test.context.junit.jupiter.SpringExtension;
import java.lang.reflect.Field;
import static org.hamcrest.CoreMatchers.equalTo;
import static org.junit.Assert.assertThat;
import static org.mockito.Mockito.when;
#ExtendWith(SpringExtension.class)
#SpringBootTest(classes = {
PreAuthConfiguration.class,
StopConfiguration.class,
})
public class FeignClientConfigTest {
#Mock
private ProviderClientConfig providerClientConfig;
#Test
#DisplayName("should return HttpClient with defaultConfig field filled with values in providerClientConfig")
public void shouldReturnHttpClientWithConfiguredValues() throws Exception {
// given
when(providerClientConfig.getConnectionRequestTimeout()).thenReturn(30000);
when(providerClientConfig.getConnectionTimeout()).thenReturn(30);
when(providerClientConfig.getMaxConnNumPerRoute()).thenReturn(20);
when(providerClientConfig.getSocketTimeout()).thenReturn(10);
FeignClientConfig feignClientConfig = new FeignClientConfig(providerClientConfig);
// when
HttpClient httpClient = feignClientConfig.buildHttpClient();
// then
// I want to test internal state of built HttpClient and this should be checked
// I tried to use PowerMock.Whitebox, but then I found it uses reflection internally
// I don't want to introduce another dependency, and PowerMock is said not to be compatible with JUnit 5, so..
Field requestConfigField = httpClient.getClass().getDeclaredField("defaultConfig");
requestConfigField.setAccessible(true);
RequestConfig requestConfig = (RequestConfig)requestConfigField.get(httpClient);
assertThat(requestConfig.getConnectionRequestTimeout(), equalTo(30000));
assertThat(requestConfig.getConnectTimeout(), equalTo(30));
assertThat(requestConfig.getSocketTimeout(), equalTo(10));
}
}
Also, I answer the first question in OP about when to test private members in a class here
Whitebox was working for me. As it is not documented here I'm adding my version:
in my case wanted to test that the timeout is different from 0 to avoid deadlock
HttpClient httpClient = factory.getHttpClient();
RequestConfig sut = Whitebox.getInternalState(httpClient, "defaultConfig");
assertNotEquals(0, sut.getConnectionRequestTimeout());
assertNotEquals(0, sut.getConnectTimeout());
assertNotEquals(0, sut.getSocketTimeout());

Getting phone call logs using API Bridge - J2ME

I'm trying to get to my nokia symbian S60 5th (NOKIA 5800) phone call logs using API Bridge. I followed the documentation from Nokia site but application doesn't work. The code is in Java ME. The problem is that I can't Initialize the API Bridge Midlet. This is the code. Thank you
package mobileapplication3;
import apibridge.*;
import apibridge.entities.*;
import com.sun.lwuit.*;
//
import java.util.*;
import java.util.Hashtable;
import java.util.Vector;
import javax.microedition.lcdui.Alert;
import javax.microedition.lcdui.AlertType;
import javax.microedition.lcdui.Command;
import javax.microedition.lcdui.CommandListener;
import javax.microedition.lcdui.Display;
import javax.microedition.lcdui.Displayable;
import javax.microedition.lcdui.Form;
import javax.microedition.location.Coordinates;
import javax.microedition.location.Criteria;
import javax.microedition.location.Location;
import javax.microedition.location.LocationException;
import javax.microedition.location.LocationProvider;
import javax.microedition.lcdui.TextBox;
//
/*import apibridge.LocationService;
import apibridge.LoggingService;
import apibridge.APIBridge;
import apibridge.HTTPManager;
import apibridge.MediaManagementService;
import apibridge.URLEncoder;
import apibridge.NewFileService;
import apibridge.entities.*;*/
import javax.microedition.midlet.*;
public class Midlet extends MIDlet implements CommandListener {
private Command exitCommand = new Command("Exit", Command.EXIT, 1);
private Command callLogCommand = new Command("Calllog", Command.ITEM, 2);
private final TextBox tbox = new TextBox("Result", "", 3000, 0);
public Midlet() {
tbox.addCommand(exitCommand);
tbox.addCommand(callLogCommand);
tbox.setCommandListener(this);
APIBridge apiBridge = APIBridge.getInstance();
apiBridge.Initialize(this);
tbox.setString("Prova ...");
}
A MIDlet doesn't get started by its constructor. It gets started by the startApp() method.
So try moving everything inside your constructor, into a function called startApp().
public Midlet() {
}
public void startApp() {
tbox.addCommand(exitCommand);
tbox.addCommand(callLogCommand);
tbox.setCommandListener(this);
APIBridge apiBridge = APIBridge.getInstance();
apiBridge.Initialize(this);
tbox.setString("Prova ...");
}
See if that helps.

Dynamic Method Invocation and JAXBElement Type on CXF

I wrote the small application below to list all the methods and of a soap service using Apache CXF library. This application lists all the methods of the service, but as it is seen on the output when you run this application, input parameters and return types of the service methods are JAXBElement for the complex types. I want cxf not to generate JAXBElement, instead I want the complex types in their original classes generated on runtime. As it is said on http://s141.codeinspot.com/q/1455881 , it can be done by setting generateElementProperty property's value to false for wsdl2java utility of cxf library, but I couldn't find the same parameter for dynamic method invocation with cxf library. I want to obtain input parameters and return types in their original types.
import java.lang.reflect.Method;
import java.net.MalformedURLException;
import java.net.URL;
import java.util.Collection;
import java.util.List;
import org.apache.cxf.binding.Binding;
import org.apache.cxf.endpoint.Client;
import org.apache.cxf.jaxws.endpoint.dynamic.JaxWsDynamicClientFactory;
import org.apache.cxf.service.model.BindingInfo;
import org.apache.cxf.service.model.BindingMessageInfo;
import org.apache.cxf.service.model.BindingOperationInfo;
import org.apache.cxf.service.model.MessagePartInfo;
import org.apache.cxf.service.model.OperationInfo;
import org.apache.cxf.service.model.ServiceModelUtil;
public class Main {
public static void main(String[] args) {
URL wsdlURL = null;
try {
wsdlURL = new URL("http://path_to_wsdl?wsdl");
} catch (MalformedURLException e) {
e.printStackTrace();
}
ClassLoader classLoader = Thread.currentThread().getContextClassLoader();
JaxWsDynamicClientFactory dcf = JaxWsDynamicClientFactory.newInstance();
Client client = dcf.createClient(wsdlURL, classLoader);
Binding binding = client.getEndpoint().getBinding();
BindingInfo bindingInfo = binding.getBindingInfo();
Collection<BindingOperationInfo> operations = bindingInfo.getOperations();
for(BindingOperationInfo boi:operations){
OperationInfo oi = boi.getOperationInfo();
BindingMessageInfo inputMessageInfo = boi.getInput();
List<MessagePartInfo> parts = inputMessageInfo.getMessageParts();
System.out.println("function name: "+oi.getName().getLocalPart());
List<String> inputParams = ServiceModelUtil.getOperationInputPartNames(oi);
System.out.println("input parameters: "+inputParams);
for(MessagePartInfo partInfo:parts){
Class<?> partClass = partInfo.getTypeClass(); //here we have input parameter object on each iteration
Method[] methods = partClass.getMethods();
for(Method method:methods){
System.out.println("method: "+method);
Class<?>[] paramTypes = method.getParameterTypes();
for(Class paramType:paramTypes){
System.out.println("param: "+paramType.getCanonicalName());
}
Class returnType = method.getReturnType();
System.out.println("returns: "+returnType.getCanonicalName());
}
System.out.println("partclass: "+partClass.getCanonicalName());
}
}
System.out.println("binding: " + binding);
}
}
Create a binding file that looks like:
<jaxb:bindings
xmlns:jaxb="http://java.sun.com/xml/ns/jaxb" jaxb:version="2.0"
xmlns:xjc="http://java.sun.com/xml/ns/jaxb/xjc" jaxb:extensionBindingPrefixes="xjc">
<jaxb:globalBindings generateElementProperty="false">
<xjc:simple />
</jaxb:globalBindings>
</jaxb:bindings>
and pass that into the JaxWsDynamicClientFactory via the createClient method that takes the List of binding files.