I have registered AsyncHornetQHTWorkItemHandler with KnowledgeRuntime
AsyncHornetQTaskClient asyncHornetQTaskClient=new AsyncHornetQTaskClient("Async-Task-ProcEngine-Client");
AsyncHornetQHTWorkItemHandler asyncHornetQHTWorkItemHandler = new AsyncHornetQHTWorkItemHandler(asyncHornetQTaskClient,knowledgeSession, OnErrorAction.RETHROW);
asyncHornetQHTWorkItemHandler.setPort(5445);
asyncHornetQHTWorkItemHandler.setIpAddress("localhost");
logger.debug("Asycnhronous work-item-handler connecting....");
asyncHornetQHTWorkItemHandler.connect();
logger.debug("Asycnhronous work-item-handler connected.");
knowledgeSession.getWorkItemManager().registerWorkItemHandler("Human Task", asyncHornetQHTWorkItemHandler);
BaseHornetQTaskServer is configured to use the standalone HornetQ server running # port 5445
public CustomHornetQTaskServer(TaskService service, String host, int port) {
super(new HornetQTaskServerHandler(service, SystemEventListenerFactory.getSystemEventListener()), host, port, true);
}
Human Tasks are created successfully in Task database when a process instance is created.
I am using AsyncHornetQTaskClient (wrapped with SyncTaskServiceWrapper) from my web application to act (complete,skip, etc) on the Tasks.
TaskService client = new SyncTaskServiceWrapper(new AsyncHornetQTaskClient("Async-Task-Client"));
client.connect("127.0.0.1", 5445);
return client;
The process engine is hosted in a separate VM to run as a daemon/background service.
Human-task service is embedded in my web-application hosted in Jboss 7.
How KnowledgeRuntime.WorkItemManager in the daemon service will get the event that work-item has completed ?
Should I explicity call KnowledgeRuntime.getWorkItemManager().completeWorkItem ?
Environment : JBPM 5.4.0.Final, HornetQ 2.2.14.Final, Jboss 7
Related
What I have:
ASP.NET Core 2.1 Web Application (self-hosted deployment, not using IIS).
When I pass --register-service or --unregister-service as command line arguments to my app, it (un)registers itself as a Windows Service and then exits.
MSI installer (WiX Toolset 3.11) which registers the web application as a Windows Service by using the above command line argument in a Custom Action.
When doing an upgrade from version 1 to version 2 of my application, the MSI installer successfully performs the following steps:
Unregister version 1 as Windows Service
Remove binaries of version 1
Copy binaries of version 2
Register and (attempt to) start version 2 as Windows Service
Problem:
The newly installed version 2 does not start properly because apparently the port is still blocked:
System.IO.IOException: Failed to bind to address https://[::]:5001: address already in use. ---> Microsoft.AspNetCore.Connections.AddressInUseException: Only one usage of each socket address (protocol/network address/port) is normally permitted ---> System.Net.Sockets.SocketException: Only one usage of each socket address (protocol/network address/port) is normally permitted
Interestingly when any of these two versions is installed, I can stop and start the Windows Service without any issues. The problem with the blocked port seems to occur specifically in the context of an MSI upgrade.
What I tried:
Once the MSI installer is finished, I can just restart the newly installed service manually and it runs fine. So it looks like it just takes some more time for the port to become available. So what I tried is some kind of a re-try mechanism, along these lines (simplified):
IWebHost host = null;
Start:
host?.Dispose();
host = CreateWebHost();
try
{
await host.RunAsync(ct);
}
catch (IOException ex) when (ex.InnerException is AddressInUseException)
{
await Task.Delay(TimeSpan.FromSeconds(5), ct);
goto Start;
}
However, then I get the following exception in some service where I try to use an injected IServiceScopeFactory when doing the first re-try:
System.ObjectDisposedException: Cannot access a disposed object.
Object name: 'IServiceProvider'.
So it looks like even I'm creating a completely new IWebHost for retrying, the DI container still seems to be in a bad state from the first try.
Questions:
What could be the reason for this blocked port, apparently only in the context of an MSI update? Could it be that the Windows Installer itself is not releasing the port? How to solve or workaround this?
With regards to the workaround I tried: Why do I get that ObjectDisposedException? Is there a better way to re-try while the port is still blocked?
I have a .net micro-service receiving messages using RabbitMQ client, I need to test the following:
1- consumer is successfully connected to rabbitMq host.
2- consumer is listening to queue.
3- consumer is receiving messages successfully.
To achieve the above, I have created a sample application that sends messages and I am debugging consumer to be sure that it is receiving messages.
How can I automate this test? hence include it in my micro-service CI.
I am thinking to include my sample app in my CI so I can fire a message then run a consumer unit test that waits a specific time then passes if the message received, but this seems like a wrong practice to me because the test will not start until a few seconds the message is fired.
Another way I am thinking of is firing the sample application from the unit test itself, but if the sample app fails to work that would make it the service fault.
Is there any best practices for integration testing of micro-services connecting through RabbitMQ?
I have built many such tests. I have thrown up some basic code on
GitHub here with .NET Core 2.0.
You will need a RabbitMQ cluster for these automated tests. Each test starts by eliminating the queue to ensure that no messages already exist. Pre existing messages from another test will break the current test.
I have a simple helper to delete the queue. In my applications, they always declare their own queues, but if that is not your case then you'll have to create the queue again and any bindings to any exchanges.
public class QueueDestroyer
{
public static void DeleteQueue(string queueName, string virtualHost)
{
var connectionFactory = new ConnectionFactory();
connectionFactory.HostName = "localhost";
connectionFactory.UserName = "guest";
connectionFactory.Password = "guest";
connectionFactory.VirtualHost = virtualHost;
var connection = connectionFactory.CreateConnection();
var channel = connection.CreateModel();
channel.QueueDelete(queueName);
connection.Close();
}
}
I have created a very simple consumer example that represents your microservice. It runs in a Task until cancellation.
public class Consumer
{
private IMessageProcessor _messageProcessor;
private Task _consumerTask;
public Consumer(IMessageProcessor messageProcessor)
{
_messageProcessor = messageProcessor;
}
public void Consume(CancellationToken token, string queueName)
{
_consumerTask = Task.Run(() =>
{
var factory = new ConnectionFactory() { HostName = "localhost" };
using (var connection = factory.CreateConnection())
{
using (var channel = connection.CreateModel())
{
channel.QueueDeclare(queue: queueName,
durable: false,
exclusive: false,
autoDelete: false,
arguments: null);
var consumer = new EventingBasicConsumer(channel);
consumer.Received += (model, ea) =>
{
var body = ea.Body;
var message = Encoding.UTF8.GetString(body);
_messageProcessor.ProcessMessage(message);
};
channel.BasicConsume(queue: queueName,
autoAck: false,
consumer: consumer);
while (!token.IsCancellationRequested)
Thread.Sleep(1000);
}
}
});
}
public void WaitForCompletion()
{
_consumerTask.Wait();
}
}
The consumer has an IMessageProcessor interface that will do the work of processing the message. In my integration test I created a fake. You would probably use your preferred mocking framework for this.
The test publisher publishes a message to the queue.
public class TestPublisher
{
public void Publish(string queueName, string message)
{
var factory = new ConnectionFactory() { HostName = "localhost", UserName="guest", Password="guest" };
using (var connection = factory.CreateConnection())
using (var channel = connection.CreateModel())
{
var body = Encoding.UTF8.GetBytes(message);
channel.BasicPublish(exchange: "",
routingKey: queueName,
basicProperties: null,
body: body);
}
}
}
My example test looks like this:
[Fact]
public void If_SendMessageToQueue_ThenConsumerReceiv4es()
{
// ARRANGE
QueueDestroyer.DeleteQueue("queueX", "/");
var cts = new CancellationTokenSource();
var fake = new FakeProcessor();
var myMicroService = new Consumer(fake);
// ACT
myMicroService.Consume(cts.Token, "queueX");
var producer = new TestPublisher();
producer.Publish("queueX", "hello");
Thread.Sleep(1000); // make sure the consumer will have received the message
cts.Cancel();
// ASSERT
Assert.Equal(1, fake.Messages.Count);
Assert.Equal("hello", fake.Messages[0]);
}
My fake is this:
public class FakeProcessor : IMessageProcessor
{
public List<string> Messages { get; set; }
public FakeProcessor()
{
Messages = new List<string>();
}
public void ProcessMessage(string message)
{
Messages.Add(message);
}
}
Additional advice is:
If you can append randomized text to your queue and exchange names on each test run then do so to avoid concurrent tests interfering with each other
I have some helpers in the code for declaring queues, exchanges and bindings also, if your applications don't do that.
Write a connection killer class that will force close connections and check your applications still work and can recover. I have code for that, but not in .NET Core. Just ask me for it and I can modify it to run in .NET Core.
In general, I think you should avoid including other microservices in your integration tests. If you send a message from one service to another and expect a message back for example, then create a fake consumer that can mock the expected behaviour. If you receive messages from other services then create fake publishers in your integration test project.
I was successfully doing such kind of test. You need test instance of RabbitMQ, test exchange to send messages to and test queue to connect to receive messages.
Do not mock everything!
But, with test consumer, producer and test instance of rabbitMQ there is no actual production code in that test.
use test rabbitMQ instance and real aplication
In order to have meaniningfull test I would use test RabbitMQ instance, exchange and queue, but leave real application (producer and consumer).
I would implement following scenario
when test application does something that test message to rabbitMQ
then number of received messages in rabbitMQ is increased then
application does something that it should do upon receiving messages
Steps 1 and 3 are application-specific. Your application sends messages to rabbitMQ based on some external event (HTTP message received? timer event?). You could reproduce such condition in your test, so application will send message (to test rabbitMQ instance).
Same story for verifying application action upon receiving message. Application should do something observable upon receiving messages.
If application makes HTTP call- then you can mock that HTTP endpoint and verify received messages. If application saves messages to the database- you could pool database to look for your message.
use rabbitMQ monitoring API
Step 2 can be implemented using RabbitMQ monitoring API (there are methods to see number of messages received and consumed from queue https://www.rabbitmq.com/monitoring.html#rabbitmq-metrics)
consider using spring boot to have health checks
If you are java-based and then using Spring Boot will significantly simpify your problem. You will automatically get health check for your rabbitMQ connection!
See https://spring.io/guides/gs/messaging-rabbitmq/ for tutorial how to connect to RabbitMQ using Spring boot.
Spring boot application exposes health information (using HTTP endpoint /health) for every attached external resource (database, messaging, jms, etc)
See https://docs.spring.io/spring-boot/docs/current/reference/html/production-ready-endpoints.html#_auto_configured_healthindicators for details.
If connection to rabbitMQ is down then health check (done by org.springframework.boot.actuate.amqp.RabbitHealthIndicator) will return HTTP code 4xx and meaninfull json message in JSON body.
You do not have to do anything particular to have that health check- just using org.springframework.boot:spring-boot-starter-amqp as maven/gradle dependency is enough.
CI test- from src/test directory
I have written such test (that connect to external test instance of RabbitMQ) using integration tests, in src/test directory. If using Spring Boot it is easiest to do that using test profile, and having details of connection to test RabbitMQ instance in application-test.properties (production could use production profile, and application-production.properties file with production instance of RabbitMQ).
In simplest case (just verify connection to rabbitMQ) all you need is to start application normally and validate /health endpoint.
In this case I would do following CI steps
one that builds (gradle build)
one that run unit tests (tests without any external dependenices)
one that run integration tests
CI test- external
Above described approach could also be done for application deployed to test environment (and connected to test rabbitMQ instance). As soon as application starts, you can check /health endpoint to make sure it is connected to rabbitMQ instance.
If you make your application send message to rabbitMQ, then you could observe rabbbitMQ metrics (using rabbitMQ monitoring API) and observe external effects of message being consumed by application.
For such test you need to start and deploy your application from CI befor starting tests.
for that scenario I would do following CI steps
step that that builds app
steps that run all tests in src/test directory (unit, integration)
step that deploys app to test environment, or starts dockerized application
step that runs external tests
for dockerized environment, step that stops docker containers
Consider dockerized enevironment
For external test you could run your application along with test RabbitMQ instance in Docker. You will need two docker containers.
one with application
one with rabbitMQ . There is official docker image for rabbitmq https://hub.docker.com/_/rabbitmq/ and it is really easy to use
To run those two images, it is most reasonable to write docker-compose file.
I'm searching for a way to get my Files synchronized (task) from a web server (Ubuntu 14) to a local server (Windows Server). The web server creates small files, which the local Server needs. The web server is in a DMZ, accessible through SSH. Only the local server is able to access folders on web server. It tried using Programs like WinSCP, but I'm not able to set a "get"-Job.
Is there a way to do this with SSH on Windows server without login every few seconds? Or is there a better solution? In the Future Web-Services are possible, but at the moment I need a quick solution.
Either you need to schedule a regular frequent job, that connects and downloads changes.
Or you need to have continuously running process, that keeps the connection opened and regularly watches for changes.
There's hardly a better solution (that's still quick and easy to implement).
Example of continuous process implemented using WinSCP .NET assembly:
// Setup session options
SessionOptions sessionOptions = new SessionOptions {
Protocol = Protocol.Sftp,
HostName = "example.com",
UserName = "user",
Password = "mypassword",
SshHostKeyFingerprint = "ssh-rsa 2048 xxxxxxxxxxx...="
};
using (Session session = new Session())
{
// Connect
session.Open(sessionOptions);
while (true)
{
// Download changes
session.SynchronizeDirectories(
SynchronizationMode.Local, localPath, remotePath, false).Check();
// Wait 10 seconds
Thread.Sleep(10000);
}
}
You will need to add a better error handling and reconnect, if connection breaks.
If you do not want to implement this as (C#) application, you can use PowerShell script. For a complete solution, see
Keep local directory up to date (download changed files from remote SFTP/FTP server).
How can i programmatically configure and host WCF Service in IIS. I have created my WCF service example /WCFServices/Service1.svc". I want to programmatically configure and host this service in IIS. Can anyone help me on this?
The class you want is Microsoft.Web.Administration.ServerManager
http://msdn.microsoft.com/en-us/library/microsoft.web.administration.servermanager(v=VS.90).aspx
It has methods for manipulating most aspects of IIS, for example, adding application pools and applications. for example, this code configures a new IIS application
//the name of the IIS AppPool you want to use for the application - could be DefaultAppPool
string appPoolName = "MyAppPool";
//the name of the application (as it will appear in IIS manager)
string name = "MyWCFService";
//the physcial path of your application
string physicalPath = "C:\\wwwroot\mywcfservice";
using (ServerManager serverManager = new ServerManager())
{
Configuration config = serverManager.GetApplicationHostConfiguration();
ConfigurationSection sitesSection = config.GetSection("system.applicationHost/sites");
ConfigurationElementCollection sitesCollection = sitesSection.GetCollection();
ConfigurationElement siteElement = sitesCollection[0];
ConfigurationElementCollection siteCollection = siteElement.GetCollection();
ConfigurationElement applicationElement = siteCollection.CreateElement("application");
applicationElement["path"] = name;
applicationElement["applicationPool"] = appPoolName;
ConfigurationElementCollection applicationCollection = applicationElement.GetCollection();
ConfigurationElement virtualDirectoryElement = applicationCollection.CreateElement("virtualDirectory");
virtualDirectoryElement["path"] = #"/";
virtualDirectoryElement["physicalPath"] = physicalPath;
applicationCollection.Add(virtualDirectoryElement);
siteCollection.Add(applicationElement);
serverManager.CommitChanges();
}
In general, the calss is just a thin wrapper around the IIS config file. You can understand it by looking at your exisiting file, or even by looking at what you have to do in IIS Manager to configure the service manually, then translating that into the resulting configuration changes.
You can do all (at least lots of) the the IIS configuration in this way (e.g. configure application throttling, enable authentication schemes etc.).
The WCF part of the configuration is just normal WCF. you can do it either in code or in configuration.
What you are looking for is called Publish. You can find it from the right-click context menu on the WCF Service project. You can publish from there or create a package for publishing later or distributing it to a remote site. There are a lot of tutorials on the net.
If you have a specific question about this feature, feel free to ask.
Have a look at msdeploy, a command line packaging and deployment tool:
http://blogs.iis.net/msdeploy/
http://vishaljoshi.blogspot.de/2009/02/web-deployment-with-vs-2010-and-iis.html
http://msdn.microsoft.com/en-us/vs2010trainingcourse_webdevelopment_topic8#_Toc282089433
My machine is Windows 7 ultimate (64 bit). I have installed MSMQ and checked that it is working fine (ran some sample codes for MSMQ).
When i try to create a WCF Service using MsmqIntegrationBinding class, i get the below exception:
"An error occurred while opening the queue:The queue does not exist or you do not have sufficient permissions to perform the operation. (-1072824317, 0xc00e0003). The message cannot be sent or received from the queue. Ensure that MSMQ is installed and running. Also ensure that the queue is available to open with the required access mode and authorization."
I am running the visual studio in Administrator mode and explicitly grant permission to myself via a URL ACL using:
netsh http add urlacl url=http://+:80/ user=DOMAIN\user
Below is the code:
public static void Main()
{
Uri baseAddress = new Uri(#"msmq.formatname:DIRECT=OS:AJITDELL2\private$\Orders");
using (ServiceHost serviceHost = new ServiceHost(typeof(OrderProcessorService), baseAddress))
{
MsmqIntegrationBinding serviceBinding = new MsmqIntegrationBinding();
serviceBinding.Security.Transport.MsmqAuthenticationMode = MsmqAuthenticationMode.None;
serviceBinding.Security.Transport.MsmqProtectionLevel = System.Net.Security.ProtectionLevel.None;
//serviceBinding.SerializationFormat = MsmqMessageSerializationFormat.Binary;
serviceHost.AddServiceEndpoint(typeof(IOrderProcessor), serviceBinding, baseAddress);
serviceHost.Open();
// The service can now be accessed.
Console.WriteLine("The service is ready.");
Console.WriteLine("The service is running in the following account: {0}", WindowsIdentity.GetCurrent().Name);
Console.WriteLine("Press <ENTER> to terminate service.");
Console.WriteLine();
Console.ReadLine();
// Close the ServiceHostBase to shutdown the service.
serviceHost.Close();
}
}
Can you please help?
Make sure you have created the "Orders" queue in MSMQ.
In Windows Server 2008, you can do so from the Server Manager (right click on My Computer and select Manage), then Features -> Message Queuing -> Private Queues. Right click on Private Queues and add your "Orders" queue there.
You may also want to check Nicholas Allen's article: Diagnosing Common Queue Errors. It suggests that your error can only be: "that the queue does not exist, or perhaps you've specified the queue name incorrectly". All the other error cases would have thrown a different exception.