I am using SQL server 2012 at sender and receiver's service broker
and i have the following contract and message on both
create MESSAGE TYPE [//Photo/Message]
VALIDATION = NONE
create CONTRACT [//photo/Contract]
([//Photo/Message] SENT BY INITIATOR)
at sender:
create SERVICE [tcp://192.168.4.173:4022/HadoopSender/Audit/HadoopDataSender]
AUTHORIZATION dbo
ON QUEUE dbo.HadoopInitiatorAuditQueue -
i have created the route at receiver as
create ROUTE [RouteDataReceiver] WITH ADDRESS = N'TRANSPORT'
when i issue send message from sender ,it reaches target server's queue but it stays at sys.transmission_queue at sender
why this message is not deleted from sender queue sys.transmission_queue
is there any configuration that i should apply ?
because the route at receiver is transport
service at sender should be renamed as following [tcp://MachineName:PORT/UniqueServiceName]
in order to let receiver service knows which IP it should send the acknowledgment to
Related
I'm building a SpringCloud Stream based application and exchange type is topic and message is sent to 2 queue consumer groups from the topic exchange. The scenario is something like this:
Service A in my application wants to send message of type appointments to service B and service C via an exchange named as: appointments-request based on different use case scenarios such as book, cancel, update etc.
So messages with a key appointments.book.B or appointments.cancel.B should go to consumer queue group appointments.B
messages with a key appointments.book.C or appointments.cancel.C should go to consumer queue group appointments.C
How to achieve this successfully?
Configuration of Producer Service:
spring.cloud.stream.bindings.output.destination=appointments-request
spring.cloud.stream.bindings.input.destination=appointments-reply
spring.cloud.stream.rabbit.bindings.output.producer.exchangeType=topic
spring.cloud.stream.rabbit.bindings.output.producer.routingKeyExpression=
appointments.#.#
Configuration of Consumer Service B:
spring.cloud.stream.rabbit.bindings.input.consumer.exchangeType=direct
spring.cloud.stream.rabbit.bindings.input.consumer.group=
appointments.docmgmt
spring.cloud.stream.rabbit.bindings.input.consumer.bindingRoutingKey=
appointments.docmgmt
spring.cloud.stream.rabbit.bindings.input.consumer.routingKeyExpression=
appointments.#.docmgmt
Producer Service A has the below method to set routing key
public boolean send(AppointmentEvent appointmentEvent)
{
logger.info("Sending event {} ",appointmentEvent);
return this.source.output().
send(MessageBuilder.withPayload(appointmentEvent).
setHeader(SimpMessageHeaderAccessor.DESTINATION_HEADER,
"appointments.book.docmgmt").build());
}
My communication between services is not working.
appointments.#.#
You can't use wildcards on the producer side.
You need something like
spring.cloud.stream.rabbit.bindings.output.producer.routingKeyExpression=headers['routingKey']
And then the producer sets the routingKey header to the desired value for each message.
You shouldn't really use the Simp headers; that is for STOMP; use your own header.
I'm trying to learn the basics of Service Broker, and have created an application originally based on the SSMS template. However I can't send a message to my queue. It just says the message type is not part of the service contract.
The absolute bare minimum I need to recreate this is the following batch:
USE [test_db]
GO
CREATE MESSAGE TYPE [test_message]
AUTHORIZATION [dbo]
VALIDATION = WELL_FORMED_XML
GO
CREATE CONTRACT [test_contract]
AUTHORIZATION [dbo] (
[test_message] SENT BY ANY
)
GO
CREATE QUEUE [dbo].[test_queue]
WITH STATUS = ON
,RETENTION = OFF
--,ACTIVATION (
-- STATUS = ON
-- ,PROCEDURE_NAME = [dbo].[test_activator]
-- ,MAX_QUEUE_READERS = 1
-- ,EXECUTE AS N'dbo'
--)
ON [PRIMARY]
GO
CREATE SERVICE [test_service]
AUTHORIZATION [dbo]
ON QUEUE [dbo].[test_queue] (
[test_contract]
)
GO
BEGIN TRANSACTION
DECLARE #dialog_handle UNIQUEIDENTIFIER
BEGIN DIALOG #dialog_handle
FROM SERVICE test_service
TO SERVICE N'test_service';
SEND ON CONVERSATION #dialog_handle
MESSAGE TYPE test_message (N'<test />');
END CONVERSATION #dialog_handle;
COMMIT TRANSACTION
GO
...which yields:
Msg 8431, Level 16, State 1, Line 10
The message type 'test_message' is not part of the service contract.
I only need to be able to send asynchronous messages within the same database. There are no remote connections to consider, and I don't strictly even need to handle a reply.
I've checked and double-checked that the message type, contract, queue and service all exist and the properties of the contract says that the message type is included
What am I missing?
As Denis already answered, you're missing the ON CONTRACT test_contract.
If you omit it then the DEFAULT contract is used. Every database has a contract named DEFAULT which has one message type, also named DEFAULT. The DEFAULT contract is used when you omit any contract in BEGIN DIALOG and the DEFAULT message type is used when you omit the message type in SEND:
BEGIN DIALOG #dialog_handle
FROM SERVICE test_service
TO SERVICE N'test_service'; <-- will use the DEFAULT contract
SEND ON CONVERSATION #dialog_handle
(N'<test />'); <-- will use the DEFAULT message type
DEFAULT message type has no validation. The DEFAULT contract binds the DEFAULT message type to both initiator and target (both can send the message in this contract, ie. SENT BY ANY). Contract and message type names are always case sensitive, irrelevant of database collation, so the name DEFAULT is case sensitive.
Try this:
BEGIN DIALOG #dialog_handle
FROM SERVICE test_service
TO SERVICE N'test_service'
ON CONTRACT test_contract;
I have got 10 queues on activemq server.
I have producer which want to push messages on one of the queue (the producer will select the queue randomly run time to put message on queue), how can I pass destination name in createProducer method.
I understand that I need to pass an object of type Destination. the producer would know the queues name on the server. Is it possible to pass (or convert) a string to Destination object type and pass that to createproducer method.
Thanks
If I understand your problem correctly;
If you're running Java and have a valid session, you could use Session.createQueue();
// Create a Destination using the queue name
Destination destination = session.createQueue("queue name");
// Create a MessageProducer from the Session to the Queue
MessageProducer producer = session.createProducer(destination);
Here is a complete example of doing this at the Apache site.
I'm working with VB.NET
So I have a server that broadcast it's own IP every second
Private Sub IPBroadcastTimer_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles IPBroadcastTimer.Tick
BroadcastIP()
End Sub
Do I have to keep broadcasting it every second for clients to pick it up after several minutes or hours?
If so, how would I set the server to listen to logins?
Here's what happens client-side and server-side:
Server : "Broadcasts IP every second" > "Listens to incoming transmissions" > "receives 3 strings [username/pass/ip]" > "Checks username and password validity on local XML file" > "sends confirmation or denial to target IP"
Client : "Listens to server broadcast" > "Receives Server IP" > "Sends 3 Strings through a form: username, password and local IP" > "Listen to server's confirmation or denial"
I used this to send, but listening doesn't seem to work...
Public EndPoint As New IPEndPoint(IPAddress.Parse("255.255.255.255"), 1001) ''Send destination port
Public EndPoint2 As New IPEndPoint(IPAddress.Any, 2001) ''Receive destination port
Public UserIPEndPoint As New IPEndPoint(IPAddress.Any, 2001)
Public UsernameEndPoint As New IPEndPoint(IPAddress.Any, 2001)
Public PassEndPoint As New IPEndPoint(IPAddress.Any, 2001)
Public Send As New UdpClient(2001) ''Send source port
Public Receive As New UdpClient(1001) '' Receive source port
Sending:
Send.Send(IPInBytes, IPInBytes.Length, EndPoint)
Receiving:
Receive.Receive(UserIPEndPoint)
How do I solve this?
What is the purpose of the broadcast? If it is for consumers to detect the service, perhaps it's better to rely on already developed protocols for this (such as Bonjour or SSDP). Like in at least SSDP, a consumer can broadcast a question ("is anybody there?") to which services may reply. Services may also spontaneously broadcast their presence, for instance when they are started, so that consumers with an ongoing detection can find them.
I'm trying to build RPC service at PHP using RabbitMQ similar to this example: http://www.rabbitmq.com/tutorials/tutorial-six-java.html
I'm using this PECL extension: http://pecl.php.net/package/amqp (version 1.0.3)
The problem is that my Callback Queue (declared at Client script) is locked for a Server when I add flag AMQP_EXCLUSIVE to it.
Here is my Server
// connect to server
$cnn = new AMQPConnection('...');
$cnn->connect();
$channel = new AMQPChannel($cnn);
// create exchange
$exchangeName = 'k-exchange';
$exchange = new AMQPExchange($channel);
$exchange->setName($exchangeName);
$exchange->setType(AMQP_EX_TYPE_DIRECT);
$exchange->declare();
// declare queue to consume messages from
$queue = new \AMQPQueue($channel);
$queue->setName('tempQueue');
$queue->declare();
// start consuming messages
$queue->consume(function($envelope, $queue)
use ($channel, $exchange) {
// create callback queue
$callbackQueue = new \AMQPQueue($channel);
$callbackQueue->setName($envelope->getReplyTo());
$callbackQueue->setFlags(AMQP_EXCLUSIVE); // set EXCLUSIVE flag
/* WARNING: Following code line causes error. See rabbit logs below:
* connection <0.1224.10>, channel 1 - error:
* {amqp_error,resource_locked,
* "cannot obtain exclusive access to locked queue 'amq.gen-Q6J...' in vhost '/'",
* 'queue.bind'}
*/
$callbackQueue->bind($exchange->getName(), 'rpc_reply');
// trying to publish response back to client's callback queue
$exchange->publish(
json_encode(array('processed by remote service!')),
'rpc_reply',
AMQP_MANDATORY & AMQP_IMMEDIATE
);
$queue->ack($envelope->getDeliveryTag());
});
And here is my Client.php
// connect to server
$cnn = new AMQPConnection('...');
$cnn->connect();
$channel = new AMQPChannel($cnn);
// create exchange
$exchangeName = 'k-exchange';
$exchange = new AMQPExchange($channel);
$exchange->setName($exchangeName);
$exchange->setType(AMQP_EX_TYPE_DIRECT);
$exchange->declare();
// create a queue which we send messages to server via
$queue = new \AMQPQueue($channel);
$queue->setName('tempQueue');
$queue->declare();
// binding exchange to queue
$queue->bind($exchangeName, 'temp_action');
// create correlation_id
$correlationId = sha1(time() . rand(0, 1000000));
// create anonymous callback queue to get server response response via
$callbackQueue = new \AMQPQueue($channel);
$callbackQueue->setFlags(AMQP_EXCLUSIVE); // set EXCLUSIVE flag
$callbackQueue->declare();
// publishing message to exchange (passing it to server)
$exchange->publish(
json_encode(array('process me!')),
'temp_action',
AMQP_MANDATORY,
array(
'reply_to' => $callbackQueue->getName(), // pass callback queue name
'correlation_id' => $correlationId
)
);
// going to wait for remote service complete tasks. tick once a second
$attempts = 0;
while ($attempts < 5)
{
echo 'Attempt ' . $attempts . PHP_EOL;
$envelope = $callbackQueue->get();
if ($envelope) {
echo 'Got response! ';
print_r($envelope->getBody());
echo PHP_EOL;
exit;
}
sleep(1);
$attempts++;
}
So in the end I just see error in RabbitMQ's logs:
connection <0.1224.10>, channel 1 - error:
{amqp_error,resource_locked,
"cannot obtain exclusive access to locked queue 'amq.gen-Q6J...' in vhost '/'",
'queue.bind'}
Question:
What is the proper way to create a callbackQueue object in a Server.php?
It appears that my Server.php has a different from Client.php connection to a RabbitMQ server. What should I do here?
How should I "share" the same (to Client.php's) connection at Server.php side.
UPDATE
Here are some more RabbitMQ Logs
My Server.php connection (Id is: <0.22322.27>)
=INFO REPORT==== 20-Jun-2012::13:30:22 ===
accepting AMQP connection <0.22322.27> (127.0.0.1:58457 -> 127.0.0.1:5672)
My Client.php connection (Id is: <0.22465.27>)
=INFO REPORT==== 20-Jun-2012::13:30:38 ===
accepting AMQP connection <0.22465.27> (127.0.0.1:58458 -> 127.0.0.1:5672)
Now I see Server.php causes error:
=ERROR REPORT==== 20-Jun-2012::13:30:38 ===
connection <0.22322.27>, channel 1 - error:
{amqp_error,resource_locked,
"cannot obtain exclusive access to locked queue 'amq.gen-g6Q...' in vhost '/'",
'queue.bind'}
My Assumption
I suspect since Client.php and Server.php do not share connection with the same Id it's impossible for them both to use exclusive queue declared in Client.php
There are a few issues with your implementation:
Exchange Declaration
Manually setting the reply queue opposed to
using a temporary queue
Use of AMQP_EXCLUSIVE in both directions
Exchange Declaration
You don't need to declare an exchange (AMQPExchange) to publish messages. In this RPC example, you need to use it as a way of broadcasting a message (e.g. temporary queue or temporary exchange). All communication will occur directly on the QUEUE and theoretically bypasses the exchange.
$exchange = new AMQPExchange($channel);
$exchange->publish(...);
QUEUEs & Reply To:
When you use AMQPQueue::setName() along with AMQPQueue::declare(), you are binding to a queue with a user defined name. If you declare the queue without a name, this is known as a temporary queue. This is useful when you need to receive a broadcasted message from a specific routing key. For this reason, RabbitMQ / AMQP generates a random temporary name. Since the queue name is made for a given instance to consume information exclusively, for its own sake, it is disposed of when the connection is closed.
When an RPC client wants to publish a message (AMQPExchange::publish()), it must specify a reply-to as one of the publish parameters. In this way, the RPC server can fetch the randomly generated name when it receives a request. It uses the reply-to name as the name of the QUEUE on which server will reply to the given client. Along with the temporary queue name, the instance must send a correlationId to ensure that the reply message it receives is unique to the request instance.
Client
$exchange = new AMQPExchange($channel);
$rpcServerQueueName = 'rpc_queue';
$client_queue = new AMQPQueue($this->channel);
$client_queue->setFlags(AMQP_EXCLUSIVE);
$client_queue->declareQueue();
$callbackQueueName = $client_queue->getName(); //e.g. amq.gen-JzTY20BRgKO-HjmUJj0wLg
//Set Publish Attributes
$corrId = uniqid();
$attributes = array(
'correlation_id' => $corrId,
'reply_to' => $this->callbackQueueName
);
$exchange->publish(
json_encode(['request message']),
$rpcServerQueueName,
AMQP_NOPARAM,
$attributes
);
//listen for response
$callback = function(AMQPEnvelope $message, AMQPQueue $q) {
if($message->getCorrelationId() == $this->corrId) {
$this->response = $message->getBody();
$q->nack($message->getDeliveryTag());
return false; //return false to signal to consume that you're done. other wise it continues to block
}
};
$client_queue->consume($callback);
Server
$exchange = new AMQPExchange($channel);
$rpcServerQueueName = 'rpc_queue';
$srvr_queue = new AMQPQueue($channel);
$srvr_queue->setName($rpcServerQueueName); //intentionally declares the rpc_server queue name
$srvr_queue->declareQueue();
...
$srvr_queue->consume(function(AMQPEnvelope $message, AMQPQueue $q) use (&$exchange) {
//publish with the exchange instance to the reply to queue
$exchange->publish(
json_encode(['response message']), //reponse message
$message->getReplyTo(), //get the reply to queue from the message
AMQP_NOPARAM, //disable all other params
$message->getCorrelationId() //obtain and respond with correlation id
);
//acknowledge receipt of the message
$q->ack($message->getDeliveryTag());
});
AMQP_EXCLUSIVE
In this case, EXCLUSIVE is only used on the Rpc client's temporary queue for each instance so that it can publish a message. In other words, the client creates a disposable temporary queue for it self to receive an answer from the RPC server exclusively. This insures no other channel thread can post on that queue. It is locked for the client and its responder only. It's important to note that AQMP_EXCLUSIVE does not prevent the RPC server from responding on the client's reply-to queue. AMQP_EXCLUSIVE pertains to two separate threads (channels instances) trying to publish to the same queue resource. When this occurs, the queue is essentially locked for subsequent connections. The same behavior occurs with an exchange declaration.
#Denis: Your implementation in this case is correct up to a point
Bad - don't re-declare the Queue in the server. That's the client's job
$callbackQueue = new \AMQPQueue($channel);
$callbackQueue->setName($envelope->getReplyTo());
$callbackQueue->setFlags(AMQP_EXCLUSIVE); // set EXCLUSIVE flag
...
$callbackQueue->bind($exchange->getName(), 'rpc_reply');
You're trying to bind to a QUEUE called tempQueue. But you've already created a queue called tempQueue in the client.php. Depending on which service is started first, the other will throw an error. So you can cut out all of that and just keep the last part:
// trying to publish response back to client's callback queue
$exchange->publish(
json_encode(array('processed by remote service!')),
'rpc_reply', //<--BAD Should be: $envelope->getReplyTo()
AMQP_MANDATORY & AMQP_IMMEDIATE
);
Then modify the above by replacing:
'rpc_reply'
with
$envelope->getReplyTo()
Don't Declare a Queue Name on the client side
// create a queue which we send messages to server via
$queue = new \AMQPQueue($channel);
//$queue->setName('tempQueue'); //remove this line
//add exclusivity
$queue->setFlags(AMQP_EXCLUSIVE);
$queue->declare();
//no need for binding... we're communicating on the queue directly
//there is no one listening to 'temp_action' so this implementation will send your message into limbo
//$queue->bind($exchangeName, 'temp_action'); //remove this line
My answer from this question replied on the RabbitMQ Official mailing list
While not using the same library here you have the official tutorials ported to PHP
https://github.com/rabbitmq/rabbitmq-tutorials/tree/master/php
The problem in your code is that you declare queues with different options.
So as one reply say, if you declare queue A as durable, then every other declaration of that queue must be durable. The same for the exclusive flag.
Also you don't need to redeclare a queue to publish messages to it. As an RPC server you assume that the address sent in the 'reply_to' property is already present. I think is the responsibility of the RpcClient to make sure the queue where it is waiting for replies exists already.
Addendum:
Exclusivity in queues means that the only the channel that declared the queue can access it.
On your server you should also declare your queue as exclusive. Remember, RabbitMQ queues should have the same flag. For example if you declare queue that is set to "durable" the other end should also declare the queue a "durable" So on your server put a flag $callbackQueue->setFlags(AMQP_EXCLUSIVE); somewhat like that from your client.