I have created a lot of error exchanges (using EasyNetQ), about 1400 items. The name of each exchange looks like ErrorExchange_xxxxx.
I want to delete them all.
Is there any script to delete the exchanges in a batch?
There is a method exchangeDelete available with Channel API
You can write a method such as below which takes list of exchangeNames to be deleted :
DELETE EXCHANGE
public void deleteExcahnges(List<String> exchangeNames){}
ConnectionFactory factory = new ConnectionFactory();
factory.setHost(localhost);
Connection connection = factory.newConnection();
Channel channel = connection.createChannel();
for(String exchangeName : exchangeNames){
channel.exchangeDelete(exchangeName);
}
}
It would delete all the exchanges.
Note:-> I am using amqp-client.jar for connecting to RabbitMQ server
Related
I'm trying to do send data to kafka after my database operation is successful.
I have a /POST endpoint which store the data in mongodb and return the whole object along with mongoDB uuid.
Now I want to perform an addition task, if data is successfully saved in mongodb i should call my kafka producer method and send the data.
Not sure how to do it.
Current Codebase
public Mono<?> createStock(StockDTO stockDTONBody) {
// logger.info("Received StockDTO body: {}, ", stockDTONBody);
Mono<StockDTO> stockDTO = mongoTemplate.save(stockDTONBody);
// HERE I WANT TO SEND TO KAFKA IF DATA IS SAVED TO MONGO.
return stockDTO;
}
Thanks #Alex for help. I
Adding my answer for others.
public Mono<?> createStock(StockDTO stockDTONBody) {
// logger.info("Received StockDTO body: {}, ", stockDTONBody);
Mono<StockDTO> stockDTO = mongoTemplate.save(stockDTONBody);
// =============== Kafka Code added======================
return stockDTO.flatMap(data -> sendToKafka(data, "create"));
}
public Mono<?> sendToKafka(StockDTO stockDTO, String eventName){
Map<String, Object> data = new HashMap<String, Object>();
data.put("event", eventName);
data.put("campaign", stockDTO);
template.send(kafkaTopicName, data.toString()).log().subscribe();
System.out.println("sending to Kafka "+ eventName + data.toString());
return Mono.just(stockDTO);
}
This can result in dual writes if your data is saved in mongo and something goes wrong while publishing to kafka. Data will be missing in kafka. Instead you should use change data capture for this. Mongo provides mongo change streams which can be used here or there are other open source kafka connectors available where you can configure the connectors to listen to changelogs of Mongo and stream those to kafka.
I'm trying to delete a scheduled job in ActiveMQ, and so far no luck.
Schedule is created among with the message using either NMS API or Amqpnetlite (except openwire lib as that one is not updated and can't be used on netstandard/netcore)
Sample code used to create the schedule with NMS, same thing done with AMQP lib:
var factory = new Apache.NMS.ActiveMQ.ConnectionFactory(brokerUri);
IConnection connection = factory.CreateConnection(user, password);
connection.Start();
ISession session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
IDestination dest = session.GetQueue(destination);
IMessageProducer producer = session.CreateProducer(dest);
producer.DeliveryMode = MsgDeliveryMode.NonPersistent;
var msg = session.CreateTextMessage("Sample text message");
msg.Properties.SetString("AMQ_SCHEDULED_CRON", "* * * * *");
producer.Send(msg);
connection.Close();
This part produces the following result in browser console, and that's what I am willing to delete:
I've read this other question and answer, also active mq system constants but no way the schedule gets deleted. Also tried to browse over the documentation but can't find anything useful so far
Does ActiveMQ even supports a programmative way of managing schedules? An AMQP solution would be great, but NMS is also appreciated.
You can manage scheduled jobs in ActiveMQ over STOMP, AMQP or simply from a JMS client. I've written about this before showing how to do it using the ActiveMQ Java client but the principle is the same. You can send messages with specific headers set that will operate on the scheduled messages.
To browse the collection of scheduled message you'd do something like the following:
Connection connection = createConnection();
Session session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE);
// Create the Browse Destination and the Reply To location
Destination requestBrowse = session.createTopic(ScheduledMessage.AMQ_SCHEDULER_MANAGEMENT_DESTINATION);
Destination browseDest = session.createTemporaryQueue();
// Create the "Browser"
MessageConsumer browser = session.createConsumer(browseDest);
connection.start();
// Send the browse request
MessageProducer producer = session.createProducer(requestBrowse);
Message request = session.createMessage();
request.setStringProperty(ScheduledMessage.AMQ_SCHEDULER_ACTION,
ScheduledMessage.AMQ_SCHEDULER_ACTION_BROWSE);
request.setJMSReplyTo(browseDest);
producer.send(request);
Message scheduled = browser.receive(5000);
while (scheduled != null) {
// Do something clever...
}
The returned messages contain the information on the actual scheduled message jobs that have been previously added. Obtaining the job Id allows you to completely cancel the delivery of said message.
To remove a scheduled message send that was scheduled using the Java client, AMQP client or other procotol client you'd do the following:
Message remove = session.createMessage();
remove.setStringProperty(ScheduledMessage.AMQ_SCHEDULER_ACTION,
ScheduledMessage.AMQ_SCHEDULER_ACTION_REMOVE);
remove.setStringProperty(ScheduledMessage.AMQ_SCHEDULED_ID,
scheduled.getStringProperty(ScheduledMessage.AMQ_SCHEDULED_ID));
producer.send(remove);
The full set of message property values that can be used when working with the scheduler are documented here, in AMQP just use the string literal of each as the Application Property value you set with the job Id to remote, or in the NMS client it'd just be a string key message property with the job Id that you want to remove.
There is one caveat though when doing this over AMQP and that would be that you'd need to ensure that the broker is using the JMS transformer ?transport.transformer=jms"see the AMQP documentation for ActiveMQ 5.
Accepted answer is working and correct. Turns out scheduledMessage.NMSMessageId was not holding the scheduler id.
Here's the clean C# source in case one is interested:
var factory = new Apache.NMS.ActiveMQ.ConnectionFactory(brokerUri);
IConnection connection = factory.CreateConnection(userName, password);
var session = connection.CreateSession(AcknowledgementMode.AutoAcknowledge);
var requestBrowse = session.GetTopic("ActiveMQ.Scheduler.Management");
var queue = session.GetQueue(queueName);
var consumer = session.CreateConsumer(queue);
connection.Start();
var producer = session.CreateProducer(requestBrowse);
var scheduledMessage = consumer.Receive(TimeSpan.FromSeconds(10));
if (scheduledMessage != null)
{
// do check with persistent storage, if schedule is canceled, remove it:
var remove = session.CreateMessage();
// get prop names from : http://activemq.apache.org/maven/apidocs/constant-values.htm
remove.Properties["AMQ_SCHEDULER_ACTION"] = "REMOVE";
remove.Properties["scheduledJobId"] = scheduledMessage.Properties.GetString("scheduledJobId");
producer.Send(remove);
}
producer.Close();
session.Close();
connection.Close();
Flow is like this: Get messages from some queue, in case some condition is met, drop the schedule entirely.
Is it possible to publish a message to only one client subscribed to channel? I know how to get the address (client list). Is it possible to get the address and publish to only that client?
In Redis you publish to a channel, not to a subscriber.
So the only way should be having one channel per client.
You can probably have one common channel to share the address, and then use the addresses as the channels name.
http://redis.io/topics/pubsub
As an example, using C# and StackExchange.Redis library:
//SERVER
var cnn = ConnectionMultiplexer.Connect("localhost");
var sub = cnn.GetSubscriber();
sub.Subscribe("common", (channel, value) =>
{
Console.WriteLine("SERVER Received in common: {0}", value);
sub.Publish("channel:" + value, "private from server to " + value);
});
//CLIENT
var cnn = ConnectionMultiplexer.Connect("localhost");
var address = "client1";
var sub = cnn.GetSubscriber();
sub.Subscribe("channel:" + address, (channel, value) =>
{
Console.WriteLine("Private: {0}", value);
});
sub.Publish("common", address);
Another solution: what I was trying to do was to have all subscribers listen but only one handle the message. The way I did this was create a stack in redis and have all subscribe to a key's list command notifications. So I would RPUSH a new value onto the key, the subscribers would get the keyspace notification and the fist to LPOP the value would handle it.
I'm new to RabbitMQ and was wondering of a good approach to this problem I'm mulling over. I want to create a service that subscribes to a queue and only pulls messages that meet a specific criteria; for instance, if a specific subject header is in the message.
I'm still learning about RabbitMQ, and was looking for tips on how to approach this. My questions include: how can the consumer pull only specific messages from the queue? How can the producer set a subject header in the message (if that's even the right term?)
RabbitMQ is perfect for this situation. You have a number of options to do what you want. I suggest reading the documentation to get a better understanding. I would suggest that you use a topic or direct exchange. Topic is more flexible. It goes like this.
Producer code connects to the RabbitMQ Broker and creates and Exchange with a specific name.
Producer publishes to exchange. Each message published will be published with a routing key.
Consumer connects to RabbitMQ broker.
Consumer creates Queue
Consumer binds Queue to the exchange, the same exchange defined in the producer. The binding also includes the routing keys for each message require for this particular consumer.
Lets say you were publishing log messages. The routing key might be something like "log.info", "log.warn", "log.error". Each message published by the producer will have the relevant routing key attached. You will then have a consumer that sends and email for all the error messages and another one that writes all the error messages to a file. So the emailer will define the binding from its queue to the exchange with the routing key "log.error". This way though the exchange receives all messages, the queue defined for the emailer will only contain the error messages. The filelogger will define a new separate queue bound to the same exchange and set up a different routing key. You could do three separate bindings for the three different routing keys require or just use the wildcard "log.*" to request all messages from the exchange starting with log.
This is a simple example that shows how you can achieve what you want to do.
look here for code examples specifically number tutorial number 5.
Making the best of exchange/routing of rabbitmq is recommended. If you do want to check according to the message content, the following code is a viable solution.
Retrieve messages from a queue and check, selectively ack the messages in which you're interested.
pull one message
GetResponse resp = channel.basicGet(QUEUE_NAME, false);
ack one message
channel.basicAck(resp.getEnvelope().getDeliveryTag(), false);
Example
import com.rabbitmq.client.*;
public class ReceiveLogs {
private final static String QUEUE_NAME = "hello";
public static void main(String[] argv) throws Exception {
ConnectionFactory factory = new ConnectionFactory();
factory.setHost("localhost");
try(Connection connection = factory.newConnection();
Channel channel = connection.createChannel();){
channel.queueDeclare(QUEUE_NAME, true, false, false, null);
// pull one message and ack manually and exit
GetResponse resp = channel.basicGet(QUEUE_NAME, false);
if( resp != null ){
String message = new String(resp.getBody(), "UTF-8");
System.out.println(" [x] Received '" + message + "'");
channel.basicAck(resp.getEnvelope().getDeliveryTag(), false);
}
System.out.println();
}
}
}
dependency
compile group: 'com.rabbitmq', name: 'amqp-client', version: '5.8.0'
To Retrieve Message from RabbitMQ we need to first connect with RabbitMQ server
public WebClient GetRabbitMqConnection(string userName, string password)
{
var client = new WebClient();
client.Credentials = new NetworkCredential(userName, password);
return client;
}
Now retrieve message from RabbitMQ using below code.
public string GetRabbitMQMessages(string domainName, string port,
string queueName, string virtualHost, WebClient client, string methodType)
{
string messageResult = string.Empty;
string strUri = "http://" + domainName + ":" + port +
"/api/queues/" + virtualHost + "/";
var data = client.DownloadString(strUri + queueName + "/");
var queueInfo = JsonConvert.DeserializeObject<QueueInfo>(data);
if (queueInfo == null || queueInfo.messages == 0)
return string.Empty;
if (methodType == "POST")
{
string postbody = "
{\"ackmode\":\"ack_requeue_true\",\"count\":
\"$totalMessageCount\",\"name\":\"${DomainName}\",
\"requeue\":\"false\",\"encoding\":\"auto\",\"vhost\" :
\"${QueueName}\"}";
postbody = postbody
.Replace("$totalMessageCount", queueInfo.messages.ToString())
.Replace("${DomainName}", domainName)
.Replace("${QueueName}", queueName);
messageResult = client.UploadString(strUri + queueName +
"/get", "POST", postbody);
}
return messageResult;
}
I think this will help you to implement RabbitMQ.
If you want to retrieve single message at a time please add the following properties with your Retrieving code .
Boolean autoAck = false;
model.BasicConsume(Queuename, autoAck);
model.BasicGet("Queuename", false);
model.BasicGet("Queuename", false);
By adding this properties of RabbitMQ you can retrieve the message one by one from the queue .Same like FIFO criteria
Hallo
I need a Tcp connection to a server alive in backgrond and applications will send data with this connection.I searched around and found WCF singleton is apporiate for this task
here is a code snippet that i use below
my question is that the good way and any problem can be with this?
string hostAddress = string.Empty;
try
{
srvHost = new ServiceHost(typeof(ControllerClass));
NetTcpBinding netTcpBinding = new NetTcpBinding(SecurityMode.None);
netTcpBinding.Security.Message.ClientCredentialType = MessageCredentialType.None;
netTcpBinding.Security.Transport.ProtectionLevel = System.Net.Security.ProtectionLevel.None;
netTcpBinding.Security.Transport.ClientCredentialType = TcpClientCredentialType.None;
srvHost.AddServiceEndpoint(typeof(IControllerContract), netTcpBinding, hostAddress);
srvHost.Credentials.WindowsAuthentication.AllowAnonymousLogons = true;
ServiceThrottlingBehavior serviceThrottlingBehavior = new ServiceThrottlingBehavior();
serviceThrottlingBehavior.MaxConcurrentCalls = 1000;
serviceThrottlingBehavior.MaxConcurrentInstances = 1000;
serviceThrottlingBehavior.MaxConcurrentSessions = 1000;
srvHost.Description.Behaviors.Add(serviceThrottlingBehavior);
srvHost.Open();
}
catch (System.TimeoutException timeoutEx)
{
Thread.Sleep(1000);
ReOpenHostConnection();//initialize again Controller Class
}
catch (Exception ex)
{
Trace.WriteLine(string.Format("cannot start Service Ex:{0}", ex.ToString()), TraceEventType.Error.ToString());
}
//Controller Class Initialize Code Snippet
TcpClient iTcpClient = new TcpClient();
iTcpClient.Connect(serverIP, serverPort);
networkStream = iTcpClient.GetStream();
aSychDataByte = new byte[iTcpClient.ReceiveBufferSize];
networkStream.BeginRead(aSychDataByte, 0, incommTcpClient.ReceiveBufferSize, ReadAsych, null);
Why do you combine TcpClient with WCF? If you need low level Tcp communication build client and server based on low level Tcp classes. If you need service and you don't bother with message formats use WCF.
For your problem. You don't need singleton. You just need the connection to be opened. For this you need to create WCF proxy instance (open channel) on the client and call the service. It will create connection to service instance wich will live until you close the client proxy, until your service host stops working or until timeout (10 minutes of client inactivity by default).