I have reader configured as below:
<bean name="reader" class="...Reader" scope="step">
<property name="from" value="#{jobParameters[from]}" />
<property name="to" value="#{jobParameters[to]}" />
<property name="pageSize" value="5"/>
<property name="saveState" value="false" /> <!-- we use a database flag to indicate processed records -->
</bean>
and a test for it like this:
#RunWith(SpringJUnit4ClassRunner.class)
#ContextConfiguration({"classpath:testApplicationContext.xml"})
#ActiveProfiles({"default","mock"})
#TestExecutionListeners( {StepScopeTestExecutionListener.class })
public class TestLeadsReader extends AbstractTransactionalJUnit4SpringContextTests {
#Autowired
private ItemStreamReader<Object[]> reader;
public StepExecution getStepExecution() {
StepExecution execution = MetaDataInstanceFactory.createStepExecution();
execution.getExecutionContext().putLong("campaignId", 1);
execution.getExecutionContext().putLong("partnerId", 1);
Calendar.getInstance().set(2015, 01, 20, 17, 12, 00);
execution.getExecutionContext().put("from", Calendar.getInstance().getTime());
Calendar.getInstance().set(2015, 01, 21, 17, 12, 00);
execution.getExecutionContext().put("to", Calendar.getInstance().getTime());
return execution;
}
#Test
public void testMapper() throws Exception {
for (int i = 0; i < 10; i++) {
assertNotNull(reader.read());
}
assertNull(reader.read());
}
Now, although the pageSize and saveState are injected correctly into my reader, the job parameters are not. According to the documentation this is all that it needs to be done and the only issues I found were about using jobParameters['from'] instead of jobParameters[from]. Any idea what could be wrong?
Also, the open(executionContext) method is not called on my reader before it enters the test method, which is not ok, because I use those job parameters to retrieve some data that needs to be available when the read method is called. This might be related to the above problem though, because the documentation concerning testing with late binding says that "The reader is initialized and bound to the input data".
You are setting the from and to as step execution context variables in your test. But in your application context configuration you are retrieving them as job parameters. You should set them as job parameters in your unit test.
Also, if you want the open/update/close ItemStream lifecycle methods to be called, you should execute the step. See http://docs.spring.io/spring-batch/trunk/apidocs/org/springframework/batch/test/JobLauncherTestUtils.html#launchStep-java.lang.String-org.springframework.batch.core.JobParameters-
Related
I have a scenario where the order of messages received at the receiving end is some times not in order.
My scenario:
MACHINE A:
I am sending a series of messages(around 12 messages) from machine A to machine B over rabbitMQ shovel.
The messages are are of different sizes. The over all size of 12 messages is close to 8mb.
Once the 12 messages are sent, i am also sending "complete message" in the end to MACHINE B.
MACHINE B:
This is the recieving machine. This machine has a single listener. It recieves all the messages sent by MACHINE A. Once it recieves "sending complete", its obvious that this is the last message from MACHINE A and MACHINE B starts processing all the messages that were recieved.
MACHINE B Configiration
<rabbit:listener-container
connection-factory="connectionFactory">
<rabbit:listener ref="onMessageCommand"
queue-names="CommandQueue" />
</rabbit:listener-container>
<bean id="onMessageCommand"
class="com.mypackage.command.messaging.OnMessageListner">
<property name="callBackObject" ref="callbackDisEvent" />
<property name="template" ref="amqpTemplate" />
</bean>
<bean id="callbackDisEvent" class="com.mypackage.command.OperationSettingsListener"></bean>
MACHINE A CODE
public void sendMessage(String messageToSend,String machineBID)
{
Message sendMessage = new Message(messageToSend, new MessageProperties());
RabbitTemplate rabbitTemplate =
messagingApplContext.getBean(RabbitTemplate.class);
rabbitTemplate.send(message, machineBID + ".command");
}
ISSUE:
Some times i am observing that one "sending complete" is recived by the MACHINE B just before some setting. Ideally "sending complete" should always be the last message recived by the MACHINE B.
May i please know what can be the issue here.
Make sure you are using message acknowledgements. More info here https://www.rabbitmq.com/confirms.html
Can you please provide some links for sample programs to implement scoped operations in the right manner. or a sample code will also suffice
The link I provided you has this example:
Collection<?> messages = getMessagesToSend();
Boolean result = this.template.invoke(t -> {
messages.forEach(m -> t.convertAndSend(ROUTE, m));
t.waitForConfirmsOrDie(10_000);
return true;
});
You just have to make sure all the sends are performed within the scope of an invoke call...
Boolean result = this.template.invoke(t -> {
...
t.send(...)
...
t.send(...)
...
return true;
}
And they will all go on the same channel. The return doesn't have to be boolean, it can be anything you want, even null.
this.template.invoke(t -> {
...
t.send(...)
...
t.send(...)
...
return null;
}
In my Managed Bootstrapper, I tried to call Engine.EvaluateCondition("MY_PROG_FOUND"); in Run() method. But it never evaluates and said something like: "This requires a running thread." and it never evaluates.
I'm trying to evaluate Bundle conditions in my managed bootstrapper but still no luck.
MY_PROG_FOUND is defined in Bundle code:
<util:RegistrySearch Id="PETRELINSTALLLOCATION"
Variable="MY_PROG_FOUND"
Root="HKLM"
Key="SYSTEM\CurrentControlSet\Control\Session Manager\Environment"
Value="$(var.my_prog_env_var)"
Result="exists"
/>
<bal:Condition Message="[WixBundleName] required the installation of My Program.">
<![CDATA[Installed OR (MY_PROG_FOUND)]]>
</bal:Condition>
How can I use Engine.EvaluateCondition() in managed bootstrapper? Does this requires to be called in a specific event, like DetectBegin()??
Any help would be really appreciated...
Thanks a bunch.
Later I figured out that the separate thread is active during event calls. So I tried it during DetectComplete event and it worked. But before that, I had to parse BootstrapperApplicationData.xml file to get WixBalCondition elements and get the collection into an array e.g. BalConditions[].
Here is my code snippet:
void BootstrapperApplication_DetectComplete(object sender, DetectCompleteEventArgs e)
{
string balCondtionMessages = string.Empty;
bool balConditionStatus = true;
foreach (var balCondition in bootAppData.BundleData.BalConditions)
{
if (!model.BootstrapperApplication.Engine.EvaluateCondition(balCondition.Condition))
{
balConditionStatus = false;
if (!String.IsNullOrEmpty(balCondtionMessages))
{
balCondtionMessages = balCondtionMessages + '\n' + "- " + balCondition.Message;
}
else
balCondtionMessages = "- " + balCondition.Message;
}
}
if (!balConditionStatus)
{
SetbalConditionMsg(balCondtionMessages);
}
}
This makes us parse for all conditions in our Bundle.wxs code, evaluate them and display the related message in our Custom Bootstrapper UI.
Using Lumia 920, it looks like my OnInvoke is never called even in Debug mode. The Constructor of ScheduledAgent that is inherited from ScheduledTaskAgent is called. Which means that the setup in WMAppManifest.xml is correct.
<Tasks>
<DefaultTask Name="_default" NavigationPage="MainPage.xaml" />
<ExtendedTask Name="BackgroundTask">
<BackgroundServiceAgent Specifier="ScheduledTaskAgent" Name="PeriodicAgent" Source="ScheduledPlaybackAgent" Type="ScheduledPlaybackAgent.ScheduledAgent" />
</ExtendedTask>
</Tasks>
Then I pretty much copied from sample code:
private void StartPeriodicAgent()
{
// Obtain a reference to the period task, if one exists
periodicTask = ScheduledActionService.Find(periodicTaskName) as PeriodicTask;
if (periodicTask != null)
{
RemoveAgent(periodicTaskName);
}
periodicTask = new PeriodicTask(periodicTaskName);
periodicTask.Description = "This demonstrates a periodic task.";
try
{
ScheduledActionService.Add(periodicTask);
}
catch (InvalidOperationException exception)
{
}
catch (SchedulerServiceException)
{
}
}
I purposely switch to Home screen after foreground app is started and waited as much as I can. Still no output or breakpoint from my ScheduledAgent::OnInvoke
Thanks!
Have you defined #define DEBUG_AGENT in ScheduledAgent.cs and included the following code in OnInvoke?
#if(DEBUG_AGENT)
ScheduledActionService.LaunchForTest(task.Name, TimeSpan.FromSeconds(30));
System.Diagnostics.Debug.WriteLine("Periodic task is started again: " + task.Name);
#endif
I found that originalPayload does not do the same as message.originalPayload in groovy expressions. That is, the following transformers do not do the same thing:
with originalPayload:
<expression-transformer evaluator="groovy" expression="originalPayload" />
with message.originalPayload:
<expression-transformer evaluator="groovy" expression="message.originalPayload" />
The following mule configuration and test case can be used to reproduce the issue:
<mule ...>
<flow name="test">
<vm:inbound-endpoint name="test.Name" path="test.Path"
exchange-pattern="request-response" />
<expression-transformer evaluator="string" expression="bar" />
<expression-transformer evaluator="groovy" expression="originalPayload" />
<!-- or alternatively -->
<!-- <expression-transformer evaluator="groovy" expression="message.originalPayload" /> -->
</flow>
</mule>
The test case can be:
#Test
public void transformers() throws MuleException {
final MuleClient client = muleContext.getClient();
final MuleMessage reply = client.send("vm://test.Path", 1, null, RECEIVE_TIMEOUT);
assertEquals(1, reply.getPayload());
}
The alternative with message.originalPayload works as expected. The one with originalPayload does not and the following exception is shown in the logs:
Exception stack is:
1. Expression Evaluator "groovy" with expression "originalPayload" returned null
but a value was required. (org.mule.api.expression.RequiredValue Exception)
What could I be doing wrong?
Thanks.
The problem is that expression-transformer and scripting:transformer use a different set of bindings than the one used by a scripting:component, and this because they call a different method in org.mule.module.scripting.component.Scriptable.
Moreover, when originalPayload gets bound, it is with the wrong value:
bindings.put("originalPayload", event.getMessage().getPayload());
Hence: MULE-6215
Following the Scripting Module Reference, they should be the same.
Also, if you take a look to org.mule.module.scripting.component.Scriptable in the scripting module, you will find that "originalPayload" is message.getPayload.
Can you post a small project reproducing the error?
public void populateBindings(Bindings bindings, MuleMessage message)
{
populateDefaultBindings(bindings);
if (message == null)
{
message = new DefaultMuleMessage(NullPayload.getInstance(), muleContext);
}
bindings.put("message", message);
//This will get overwritten if populateBindings(Bindings bindings, MuleEvent event) is called
//and not this method directly.
bindings.put("payload", message.getPayload());
//For backward compatability
bindings.put("src", message.getPayload());
}
public void populateBindings(Bindings bindings, MuleEvent event)
{
populateBindings(bindings, event.getMessage());
bindings.put("originalPayload", event.getMessage().getPayload());
bindings.put("payload", event.getMessage().getPayload());
bindings.put("eventContext", new DefaultMuleEventContext(event));
bindings.put("id", event.getId());
bindings.put("flowConstruct", event.getFlowConstruct());
if (event.getFlowConstruct() instanceof Service)
{
bindings.put("service", event.getFlowConstruct());
}
}
I'm a newbie in Spring Batch. I have inherited a batch process implemented with Spring Batch.
This works well, except for one thing I'll try to describe.
I launch parseJob and, when it's reading XML to process in bean parsingStepReader,
read() method is been invoking always.
The directory *path_to_xml* contains only one XML, invoke read() and return XML parsed, which is processed OK. Then, read() method is invoked again, return a null object, and is invoked again, return null... and so on.
When debugging, MultiResourceItemReader read method try to read, does not read anything (all resources has already been readed), increment currentResources and return null.
I have readed something about the job stops when the reader return a null object, but that read method returns null and reads again and again...
I changed restartable to false, but does not work.
The job is launched in Linux, batch mode, with org.springframework.batch.core.launch.support.CommandLineJobRunner
Because of this problem, the .sh that launch the job does not finish, and resources are busy.
How can I avoid this, or stop the job when resources (XML) input directory have already been processed?
Any help would be very appreciated. Best regards.
Beans file and Java class pieces are attached
<batch:job id="parseJob" restartable="true" incrementer="jobParametersIncrementer">
<batch:flow parent="parseFlow"/>
<batch:flow .../>
<batch:flow .../>
</batch:job>
<batch:flow id="parseFlow">
<batch:step id="parsingStep">
<batch:tasklet start-limit="100" allow-start-if-complete="true" transaction-manager="..." task-executor="taskExecutor" throttle-limit="$...">
<batch:chunk reader="parsingStepReader" writer="..." processor="..." commit-interval="..." skip-limit="10000000">
<batch:skippable-exception-classes>
<batch:include class="java.lang.Exception" />
</batch:skippable-exception-classes>
</batch:chunk>
<batch:listeners>
<batch:listener ref="iwListener" />
<batch:listener ref="mySkipListener" />
<batch:listener ref="myStep1Listener" />
</batch:listeners>
<batch:no-rollback-exception-classes>
<batch:include class="java.lang.Exception" />
</batch:no-rollback-exception-classes>
</batch:tasklet>
</batch:step>
</batch:flow>
<!-- -->
<bean id="bpfReader" class="org.springframework.batch.item.xml.StaxEventItemReader" scope="prototype">
<property name="fragmentRootElementName" value="..." />
<property name="unmarshaller" ref="..." />
<property name="strict" value="false" />
</bean>
<bean id="multiresourceItemReader" class="...SyncMultiResourceItemReader" abstract="true">
<property name="strict" value="false" />
<property name="delegate" ref="bpfReader" />
</bean>
<bean id="parsingStepReader" parent="multiresourceItemReader" scope="step">
<property name="resources" value="<path_to_xml>" />
</bean>
And the reader class is:
public class SyncMultiResourceItemReader<T> extends MultiResourceItemReader<T> {
. . .
#Override
public T read() throws Exception, UnexpectedInputException, ParseException {
synchronized (this) {
return super.read();
}
}
. . .
}
UPDATE: Solution suggested by #vsingh works perfectly. Once an input element is chosen, it must be removed from the input. I don't know why, but class org.springframework.batch.item.file.MultiResourceItemReader does not work as I expected, especially in an input error.
I hope this helps. Best regards
The read method will read the data , store at class level and pass it to the write method.
I will give you an example of how we did it
for eg
#Override
public Long read() throws Exception, UnexpectedInputException,
ParseException, NonTransientResourceException {
synchronized (this.myIds) {
if (!this.myIds.isEmpty()) {
return this.myIds.remove(0);
}
return null;
}
}
myIds is a List at class level
This list is populated at before step method
#Override
public void beforeStep(final StepExecution stepExec) {
this.stepExecution = stepExec;
// read the ids from service and set at class level
}
Solution suggested by #vsingh works perfectly. Once an input element is chosen, it must be removed from the input. I don't know why, but class org.springframework.batch.item.file.MultiResourceItemReader does not work as I expected, especially in an input error.
I hope this helps. Best regards