I have two kinds of workers for a same event.
I would like a message be dispatched to only one among some of my workers (like "direct" exchanges). But the other workers should all process the message (like fanout).
It's a bit hard to explain but the idea is here. And maybe the following schema will help you to understand what I would like.
Do you have a solution?
Kind regards,
Ben
If I understand you correctly, you would like to have "type-1" as worker where only a single worker work on an item while "type-2" can be treated as multiple handlers (like log-handlers) where all should accept the event.
If i'm right , then you might be able to chain two queues (exchanges) .
exchange1 - fanout - all the "type-2" (loggers) will wait here so they will all get the event
exchange2 - direct - all your "type-1" will wait here so only one will get the event
the trick - you need to make sure that you have a consumer listening on "exchange1" that will also publish to "exchange2".
your best option is to use routing keys with multiple bindings between your exchange and your queues.
i would recommend either direct or topic exchange for this, but not fanout.
to model your example image above, your configuration would look like this:
| exchange | routing key | queue |
|----------|-------------|---------|
| some.ex | type.1 | queue.1 |
| some.ex | type.1 | queue.2 |
| some.ex | type.1 | queue.3 |
| some.ex | type.2 | queue.4 |
| some.ex | type.2 | queue.5 |
basically, you need to have a routing key per queue and a queue per worker.
you may want to read up a bit more on exchanges, queues and bindings, to get a better understanding of when they would be used. i have a few ebooks that cover this (along with other RMQ usage scenarios) at https://leanpub.com/u/derickbailey
Related
I'm trying to work out how the hexagonal (ports and adapters) architecture might be used in the context of an embedded software system.
if I understand right, the architecture is something like this.
/-----------------\ /-----------------------------\
| | | |
| Application | | Domain |
| | | |
| +----------+ | | +---------+ |
| | +-------------->|interface| | /-------------------\
| +----------+ | | +---------+ | | |
| | | ^ | | Infrastructure |
| | | | | | |
\---------------+-/ | +---+---+ +---------+ | | +----------+ |
| | +---->|interface|<-------------+ | |
Code that allows | +-------+ +---------+ | | +----------+ |
interaction with | | | |
user \--------------------------+--/ \-----------------+-/
Business logic What we (the business)
depend on - persistence,
crypto services etc
Let's take a concrete example of where one of the user interfaces is a touch screen that the main controller talks to over a serial UART. The software sends control commands to draw elements on the screen and the user actions generate text messages.
The bits I see working in this scenario are:
Serial driver
Sends data over the UART
Receives data (an ISR is invoked)
Screen Command builder
Screen Response/Event parser
Business logic such as presenting and responding to menus, widgets etc
The bit I'm struggling with is where these pieces should reside. Intuitively, I feel it's as follows:
Infrastructure - UART driver
Domain - Business logic
Application - Message builder/parser
But this arrangement forces a dependency between Application and Infrastructure where the parser needs to retrieve the data and the builder needs to send the data through the UART.
Bringing the messages builder and parser to Infrastructure or Domain takes the whole user interaction thing away from the Application.
Whichever way I look at it, it seems to violate some aspect of the diagram that I drew above. Any suggestions?
I want to design a rounding policy for several components:
// | exchange | type | routing key | queue |
// |--------------|-------|----------------------------|------------------------------|
// | processing | topic | processing.trx.elavon | processing-elavon-sale |
// | processing | topic | processing.trx.elavon | processing-elavon-authorize |
// | processing | topic | processing.trx.elavon | processing-elavon-capture |
// | processing | topic | processing.trx.genesis | processing-genesis-sale |
I tried with Topic exchange and Direct exchange but I get error Reply received after timeout. After investigation it turns out that I'm sending messages to 1 queue but 2 queues are receding messages.
My caseI would like to use routing key and as additional level or abstraction as configuration variable for the target gateway and then to send the type of message to a queue.
But looks like this design is not appropriate.
What routing policy should be more suitable in order to avoid the error: Reply received after timeout? Only one queue should receive the message and reply.
EDIT:
In my case I use Spring AMQP to send messages:
convertSendAndReceive(
ContextServer.EXCHANGE_PROCESSING, ContextServer.ROUTING_KEY_PROCESSING_TRANSACTION_ELAVON, "some_payload");
But how I can reach just one queue processing-elavon-sale, not all queues bind to routing key processing.trx.elavon?
Hi all, hoping someone can assist me with some queries/configuration for the use of the Geode Redis Adapter. I'm having some difficulty ascertaining how/whether I can configure a number of Redis servers within my Geode cluster to function in a high availability setup.
I'm very new to Geode, but understand that in a traditional Geode application, the client interacts with a locator process to access data from the cluster and balance load. Given that the aim of this adapter is to function as a drop-in replacement for Redis (i.e. no change required on the client) I imagine it functions somewhat differently.
Here is what I have tried so far:
I have built from source according to this link and successfully got the gfsh cli up on 2 CentOS 7 VMs:
192.168.0.10: host1
192.168.0.15: host2
On host1, I run the following commands:
gfsh>start locator --name=locator1 --bind-address=192.168.0.10 --port=10334
gfsh>start server --name=redis --redis-bind-address=192.168.0.10 --redis-port=11211 --J=-Dgemfireredis.regiontype=PARTITION_REDUNDANT
On host2, I run the following command:
gfsh>start server --name=redis2 --redis-bind-address=192.168.0.15 --redis-port=11211 --J=-Dgemfireredis.regiontype=PARTITION_REDUNDANT --locators=192.168.0.10[10334]
On host1, I examine the current configuration:
gfsh>list members
Name | Id
-------- | -------------------------------------------------
locator1 | 192.168.0.10(locator1:16629:locator)<ec><v0>:1024
redis2 | 192.168.0.15(redis2:6022)<ec><v2>:1024
redis | 192.168.0.10(redis:16720)<ec><v1>:1025
gfsh>list regions
List of regions
-----------------
__HlL
__ReDiS_MeTa_DaTa
__StRiNgS
For each of the regions, I can see both server redis & redis2 as Hosting Members - e.g.
gfsh>describe region --name=__StRiNgS
..........................................................
Name : __StRiNgS
Data Policy : normal
Hosting Members : redis2
redis
Non-Default Attributes Shared By Hosting Members
Type | Name | Value
------ | ----- | -----
Region | size | 0
| scope | local
At this point, I turned to the redis-cli for some testing. Given the previous output, my expectation was that if I set a key on one server, I should be able to read it back from the other server:
192.168.0.10:11211> set foo 'bar'
192.168.0.10:11211> get foo
"bar"
192.168.0.15:11211> get foo
(nil)
gfsh>describe region --name=__StRiNgS
..........................................................
Name : __StRiNgS
Data Policy : normal
Hosting Members : redis2
redis
Non-Default Attributes Shared By Hosting Members
Type | Name | Value
------ | ----- | -----
Region | scope | local
Non-Default Attributes Specific To The Hosting Members
Member | Type | Name | Value
------ | ------ | ---- | -----
redis2 | Region | size | 0
redis | Region | size | 1
As you can see, a query against host2 for the key added on host1 returned (nil). I'd greatly appreciate any help here. Is it possible achieve what I'm aiming for here or does the Redis adapter only allow you to scale out a single server?
This may not be an answer, but it is probably too long for a comment.
I am not familiar with the specific Geode Redis Adapter you are talking about here. But from my experience with Gemfire/Geode, there are things you may want to check:
You started the first host without locators param, I am not sure whether this will cause any problem with cluster formation. There are two ways in Gemfire to form a cluster: by specifying mcast port or by specifying locators.
Scope of the region you are inspecting looks wrong. "local" will not replicate any updates. When you set it up correctly, it should show up as DISTRIBUTED_NO_ACK / DISTRIBUTED_ACK / GLOBAL I suppose.
Hope this helps
Xiawei is correct, a scope "local" regions will not replicate the entry on other members. The workaround for this could have been to just create a region named __StRiNgS from gfsh, but since region names starting with two underscores are for internal use only, that's not possible.
I have filed this issue https://issues.apache.org/jira/browse/GEODE-1921 to fix the problem. I also attached a patch for this issue. With the patch applied I see that the __StRiNgS region now is PARTITION.
gfsh>start locator --name=locator1
gfsh>start server --name=redis --redis-port=11211
gfsh>list regions
List of regions
-----------------
HlL
StRiNgS
__ReDiS_MeTa_DaTa
gfsh>describe region --name=/StRiNgS
..........................................................
Name : StRiNgS
Data Policy : partition
Hosting Members : redis
Non-Default Attributes Shared By Hosting Members
Type | Name | Value
------ | ----------- | ---------
Region | size | 0
| data-policy | PARTITION
On host1:
start locator --name=locator1 --bind-address=192.168.0.10 --port=10334
start server --name=redis --redis-bind-address=192.168.0.10 --redis-port=11211 --J=-Dgemfireredis.regiontype=REPLICATE
NOTE: Yoy have to use regiontype as "REPLICATE" if you wish the data to be replicated from one region to another.
On host2:
start server --name=redis2 --redis-bind-address=192.168.0.15 --redis-port=11211 --J=-Dgemfireredis.regiontype=REPLICATE --locators=192.168.0.10[10334]
https://geode.apache.org/docs/guide/11/developing/region_options/region_types.html
we're going to use rabbitmq in our project, but facing a problem that, we want to debug on our dev machines, so the response message have to be send to machine which originally send the request message out. How we're going to achive that, is there an existing solution in spring-rabbitmq framework?
We have considered several solutions. such as declare a set of queues for each machine, the queue name prefix by machine name. Is that feasible?
Define set of queues (debug queue A-Z) and bind them to headers exchange with attributes x-match=any, from=[A-Z], to=[A-Z] respectively to . Then bind headers exchange to you main working exchange (one or more) to receive all messages you interested in, so when your consumer publish response it will be duplicated to your debug exchange and then routed to appropriate queue.
[sender X] [ worker ] [consumer on queue X]
| ^ |
[request] | [response from=X, to=X] [duped request from=X|
\ | | [duplicated response from=X, to=X]
\ [request from=X] | ^
v | V |
[working topic exchange] -------> [debug headers exchange]
/ | \ / | \
{bindings by routing key mask} {bindings by any headers from=[A-Z], to=[A-Z]}
/ | \ / | \
[working queue 1] ... [working queue N] [debug queue A] ... [debug queue Z]
To bind request and response messages you can use applicationId and correlationId message attributes.
Note, that both request and response messages will be duplicated to debug queues. You may also specify separate queue for request and response messages by binding queues to match only specific headers, something like x-match=all, from=[A-Z] or x-match=all, to=[A-Z] and publish response and request messages with only that headers (only from or only to), but it is up to you.
The pros:
easy to implement
requires minimal code changes
easy to turn on/off
may be safely run in production environment
Cons:
use more resource power from RabbitMQ side
Alternatively, you can utilize RPC pattern somehow if you debugging process requires realtime response receiving. But this will block publisher until response processed, which may differ from real-world app usage and break business logic.
Pros:
step-by-step debugging process
Cons:
hard to implement
may require a lot of code changes
break business logic
hard to enable/disable
not production environment safe
p.s.: sorry for ascii graph
I've got three XBees. 2x PROs and a standard, all series 2s.
I've configured one PRO and one standard to be router/endpoints on channel 0 and PAN 234 (this is the default channel and PAN ID when selecting the "XBP24-B"/"XB24-B" "ZNET2.5 ROUTER/END DEVICE AT" profile (version 1247 for both).
However the one PRO I've set with the "ZNET 2.5 COORDINATOR AT" profile has a channel of E (though if I keep flashing the device with the same profile, this changes from 12-F).
Obviously if the coordinator doesn't have the same channel, nothing will work, but I can't see any way of setting the channel manually..?
The CH setting in X-CTU is read only, and I can see any other UI element to change the channel:
I've even been into the terminal and typed (words in brackets are what the terminal returns):
+++ (OK)
ATCH (E)
ATCH0 (ERROR)
ATCH 0 (ERROR)
ATCH00 (ERROR)
ATCH 00 (ERROR)
ATCH E (ERROR)
ATCHE (ERROR)
I've Googled and Googled to no avail. incredibly frustrating, can anyone help?!
I've had them working previous as a matter of fluke as I kept flashing the hardware until the channel numbers match up, but this is obviously ridiculous!
Channel selection with the XBee ZB (S2, S2B, S2C) series of modules works differently than with the XBee 802.15.4 (S1) modules. Channel selection is automatic with ZB (as opposed to it being manual with the 802.15.4 modules).
You normally never need to manipulate the channel selection parameters with ZB. Modules find each other and associate with each other if they can.
If your modules just can't seem to find each other it usually comes down to a mismatch in the PAN settings (ID), security settings (LK), or network joining permission settings on the coordinator (NJ).
Not a lot of information exists on the web outside of the Digi's XBee ZB OEM manual. For reference sake, channel selection with ZB works like this:
XBee ZB Coordinator is powered up
The XBee ZB Coordinator reads its SC parameter and builds a list of candidate channels to scan
The XBee ZB Coordinator then performs an energy scan on each candidate channel
The XBee ZB Coordinator then chooses the channel with the least amount of energy on it
This procedure aims to pick a channel with the least amount of noise on it be it from microwave ovens, WiFi networks, or anything else that might be transmitting on the 2.4GHz frequency band.
Any router or end devices joining a network with consult their SC parameters first, then they will try and search for networks they can join which match their PAN and security parameters. They will join and stay joined to the first network they can--with some minor exceptions (see the JV and NW parameters, for example).
If you want to force a channel selection you must set the SC parameter to enable only a single channel. The SC parameter is a bitmask1. Each bit set in the mask will enable one additional channel. What's tricky about this parameter is that the first bit (bit 0) is not channel 0, it's channel 11 (0x0B). For ease of use, if you wanted to lock an XBee ZB to a single channel here would be the values:
+---------------+---------------+------------------+-------------------------+
| Channel (Dec) | Channel (Hex) | XBee ZB SC Value | XBee Availability |
+---------------+---------------+------------------+-------------------------+
| 11 | 0xB | 0x1 | All |
| 12 | 0xC | 0x2 | All |
| 13 | 0xD | 0x4 | All |
| 14 | 0xE | 0x8 | All |
| 15 | 0xF | 0x10 | All |
| 16 | 0x10 | 0x20 | All |
| 17 | 0x11 | 0x40 | All |
| 18 | 0x12 | 0x80 | All |
| 19 | 0x13 | 0x100 | All |
| 20 | 0x14 | 0x200 | All |
| 21 | 0x15 | 0x400 | All |
| 22 | 0x16 | 0x800 | All |
| 23 | 0x17 | 0x1000 | All |
| 24 | 0x18 | 0x2000 | All |
| 25 | 0x19 | 0x4000 | S1, S2B, S2C (not S2) |
| 26 | 0x1A | 0x8000 | S1 only |
+---------------+---------------+------------------+-------------------------+
Obviously if the coordinator doesn't
have the same channel, nothing will
work, but I can't see any way of
setting the channel manually..?
I'm not sure if what you say above is right. From http://ftp1.digi.com/support/documentation/90000976_C.pdf (you should look at the correct version for your hardware, though), it looks like the purpose of a coordinator is to automatically determine the channel:
Coordinator Operation
Forming a Network
The coordinator is responsible for
selecting the channel, PAN ID (16-bit
and 64-bit), security policy, and
stack profile for a network. Since a
coordinator is the only device type
that can start a network, each ZigBee
network must have one coordinator.
After the coordinator has started a
network, it can allow new devices to
join the network. It can also route
data packets and communicate with
other devices on the network. To
ensure the coordinator starts on a
good channel and unused PAN ID, the
coordinator performs a series of scans
to discover any RF activity on
different channels (energy scan) and
to discover any nearby operating PANs
(PAN scan). The process for selecting
the channel and PAN ID are described
in the following sections.
Channel Selection
When starting a
network, the coordinator must select a
"good" channel for the network to
operate on. To do this, it performs an
energy scan on multiple channels
(frequencies) to detect energy levels
on each channel. Channels with
excessive energy levels are removed
from its list of potential channels to
start on.
I've actually never used the Digi XBee radios (just some of their other radios), so I don't know much about coordinators. I think that their user interface typically does expose all of the valid commands, so it probably won't work to try sending them manually (as you discovered).
You may find more experts on the Digi forums.
Hmm. strange, keep going over the settings again and finally got it working with 2-way comms?! :S
Setup one a coordinator
Setup other two as router/end devices
Set the Device High (DH) to 0 and the Device Low (DL) to FFFF (this means everything sent from this module should be received by everyone)
Set Negotiate Channel to 1 (Enabled), which means the only time a router/end device can set it's own channel is when it finds a coordinator on the same channel (this clearly wasn't happening in my case)
Everything else was left as default.
Leave coordinator powered on when configuring router/end devices so you can check they pair correctly.
As I said, I've used this configuration before and it just didn't work, so I don't know what kicked it into life this time, but it worked?!
One thing to take into account is that ZigBee channels extend from 11 to 26, zero is not a valid option.
Since that's the case, are you able to try the command ATCH11 ?
If that command succeeds, then perhaps the ATCH command wants a decimal input between 11 and 26?