I'm trying to implement HA policy for messaging on domain, but I have a problem with the slave-host.
This is the log:
"WFLYCTL0030: No resource definition is registered for address [
(\"subsystem\" => \"messaging-activemq\"),
(\"server\" => \"clustermqfse\"),
(\"jms-topic\" => \"test2\")
]"
The host-master, on the other hand, is ok. I saw the host.xml of both and are egual.
How could I resolve the issue?
Have you defined a topic with the name=test2 under the activemq subsystem config of the ha profile in domain.xml?
Related
I'm trying to migrate a docker based redis container into AWS Elasticache. I have the Redis instance running and can connect via the redis CLI but when I setup the logstash with the following:
input {
redis {
host => "redis<domain>.cache.amazonaws.com"
data_type => "list"
key => "logstash"
codec => msgpack
}
}
It explodes with this:
[2022-02-02T13:52:27,575][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/share/logstash/logstash.conf"], :thread=>"#<Thread:0x547a32a1 run>"}
[2022-02-02T13:52:28,685][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.11}
[2022-02-02T13:52:28,701][INFO ][logstash.inputs.redis ][main] Registering Redis {:identity=>"redis://#redis<domain>.cache.amazonaws.com:6379/0 list:logstash"}
[2022-02-02T13:52:28,709][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-02-02T13:52:28,823][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-02-02T13:52:28,837][ERROR][logstash.inputs.redis ][main][08c8cf37082e202fd617f2bc3c642b630c437b5e58521b08cd412f29ed9a10e1] Unexpected error {:message=>"invalid uri scheme ''", :exception=>ArgumentError, :backtrace=>["/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/redis-4.5.1/lib/redis/client.rb:473:in `_parse_options'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/redis-4.5.1/lib/redis/client.rb:94:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/redis-4.5.1/lib/redis.rb:65:in `initialize'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-redis-3.7.0/lib/logstash/inputs/redis.rb:129:in `new_redis_instance'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-redis-3.7.0/lib/logstash/inputs/redis.rb:134:in `connect'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-redis-3.7.0/lib/logstash/inputs/redis.rb:186:in `list_runner'", "org/jruby/RubyMethod.java:131:in `call'", "/usr/share/logstash/vendor/bundle/jruby/2.5.0/gems/logstash-input-redis-3.7.0/lib/logstash/inputs/redis.rb:87:in `run'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:409:in `inputworker'", "/usr/share/logstash/logstash-core/lib/logstash/java_pipeline.rb:400:in `block in start_input'"]}
but when I then use this configuration to provide the uri:
input {
redis {
host => "redis://redis<domain>.cache.amazonaws.com"
data_type => "list"
key => "logstash"
codec => msgpack
}
}
I get this:
[2022-02-02T13:57:10,475][INFO ][logstash.javapipeline ][main] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>1, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.max_inflight"=>125, "pipeline.sources"=>["/usr/share/logstash/logstash.conf"], :thread=>"#<Thread:0x20738737 run>"}
[2022-02-02T13:57:11,586][INFO ][logstash.javapipeline ][main] Pipeline Java execution initialization time {"seconds"=>1.11}
[2022-02-02T13:57:11,600][INFO ][logstash.inputs.redis ][main] Registering Redis {:identity=>"redis://#redis://redis<domain>.cache.amazonaws.com:6379/0 list:logstash"}
[2022-02-02T13:57:11,605][INFO ][logstash.javapipeline ][main] Pipeline started {"pipeline.id"=>"main"}
[2022-02-02T13:57:11,724][INFO ][logstash.agent ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2022-02-02T13:57:11,843][WARN ][logstash.inputs.redis ][main][08c8cf37082e202fd617f2bc3c642b630c437b5e58521b08cd412f29ed9a10e1] Redis connection error {:message=>"Error connecting to Redis on redis://redis<domain>.cache.amazonaws.com:6379 (SocketError)", :exception=>Redis::CannotConnectError}
The latter error looks saner but the Registering Redis line looks messed up. But neither provide any insight as to why they can't connect, yet I can connect to the Redis instance from the pod. What am I missing here?
Turned out on top of the config, I also had an environment variable set called REDIS_URL that was trying to gazump the config as it's used in the Redis client.
From the readme, I finally discovered:
By default, the client will try to read the REDIS_URL environment
variable and use that as URL to connect to. The above statement is
therefore equivalent to setting this environment variable and calling
Redis.new without arguments.
I'm trying to connect a Logstash instance, running in a docker container, to an Amazon MQ Broker.
My ultimate goal is to insert the MQ message bodies into ElasticSearch
Based on the logs, I think Logstash is able to reach the MQ Queue, but the error message doesn't give any other info:
[2021-05-21T23:30:53,226][ERROR][logstash.inputs.rabbitmq ][instance_journal_pipeline][rmq_instance_source]
RabbitMQ connection error, will retry. {
:error_message=>"An unknown error occurred. RabbitMQ gave no hints as to the cause. Maybe this is a configuration error (invalid vhost, for example). I recommend checking the RabbitMQ server logs for clues about this failure.",
:exception=>"Java::JavaIo::IOException"
}
My input config is as follows:
input {
rabbitmq {
id => "rmq_instance_source"
ack => true
durable => true
passive => true
exchange => "events"
exchange_type => "topic"
host => "${AWS_MQ_URL}"
user => "${AWS_MQ_USER}"
port => "${AWS_MQ_PORT}"
password => "${AWS_MQ_PASSWORD}"
queue => "outbound_task_queue_name"
key => "outbound_task_key"
arguments => {
# arguments reproduced from the RMQ Queue's admin page
}
}
}
Turns out that this was an SSL configuration issue. The RMQ Logstash plugin requires you to specify that the server is using SSL, and version. The error message does not tell you that this is the case. Verify your SSL version with your MQ Broker.
I added the following params to my config (to match the Amazon MQ configuration) and that solved it:
ssl => true
ssl_version => "TLSv1.2"
Rabbit MQ set up in my organization uses LDAP for Authenticaton and Authorization.
How can I configure NServiceBus (or RabbitMQ) to use the credentials that the service is running under (- like integrated security for SQL Connections).
Rabbmit MQ Configuration
[
{rabbit,
[{auth_backends, [rabbit_auth_backend_ldap]}]},
{rabbitmq_auth_backend_ldap,
[ {servers, ["ad.xxxx.xxx"]},
{dn_lookup_attribute, "userPrincipalName"},
{dn_lookup_base, "OU=xxxx Users,DC=ad,DC=xxxx,DC=xxx"},
{log, true},
{group_lookup_base, "OU=xxxx Users,DC=ad,DC=xxxx,DC=xxx"},
{tag_queries, [{administrator, {in_group, "CN=GRP_Name,OU=XXXX Users,DC=ad,DC=XXXX,DC=XXX"}},
{management, {in_group, in_group, "CN=GRP_Name,OU=XXXX Users,DC=ad,DC=XXXX,DC=XXX"}}]}
]
}
].
NServiceBus Code:
var endpointConfiguration = new EndpointConfiguration("Receiver.Service");
var transport = endpointConfiguration.UseTransport<RabbitMQTransport>();
transport.UseConventionalRoutingTopology();
transport.ConnectionString("host=rabbitmq.sb.xxxx.xxx");
RabbitMQ's LDAP support requires that client applications pass a username and password. There is no equivalent to SQL's integrated security.
In your case, user's must have a DN whose value ends with OU=xxxx Users,DC=ad,DC=xxxx,DC=xxx. Your NServiceBus application will have to pass a username and password of an account with the expected DN.
https://www.rabbitmq.com/ldap.html
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
I am trying to setup a logstash configuration to push lines from a file to rabbitmq. I installed both logstash 2.1.1 and rabbit 3.6.0 on my local machine to test it. the output configuration is:
output {
rabbitmq {
exchange => "test_exchange"
key => "test"
exchange_type => "direct"
host => "127.0.0.1"
port => "15672"
user => "logstash"
password => "logstashpassword"
}
}
But when I now start logstash it fails to startup with the following error (the output is only shown when logstash is started in debug mode):
Worker threads expected: 4, worker threads started: 4 {:level=>:info, :file=>"logstash/pipeline.rb", :line=>"186", :method=>"start_filters"}
Connecting to RabbitMQ. Settings: {:vhost=>"/", :host=>"127.0.0.1", :port=>15672, :user=>"logstash", :automatic_recovery=>true, :pass=>"logstashpassword", :timeout=>0, :heartbeat=>0} {:level=>:debug, :file=>"logstash/plugin_mixins/rabbitmq_connection.rb", :line=>"135", :method=>"connect"}
The error reported is:
com.rabbitmq.utility.BlockingCell.get(com/rabbitmq/utility/BlockingCell.java:77)
com.rabbitmq.utility.BlockingCell.uninterruptibleGet(com/rabbitmq/utility/BlockingCell.java:111)
com.rabbitmq.utility.BlockingValueOrException.uninterruptibleGetValue(com/rabbitmq/utility/BlockingValueOrException.java:37)
com.rabbitmq.client.impl.AMQChannel$BlockingRpcContinuation.getReply(com/rabbitmq/client/impl/AMQChannel.java:367)
com.rabbitmq.client.impl.AMQConnection.start(com/rabbitmq/client/impl/AMQConnection.java:293)
com.rabbitmq.client.ConnectionFactory.newConnection(com/rabbitmq/client/ConnectionFactory.java:621)
com.rabbitmq.client.ConnectionFactory.newConnection(com/rabbitmq/client/ConnectionFactory.java:648)
java.lang.reflect.Method.invoke(java/lang/reflect/Method.java:606)
RUBY.new_connection_impl(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare/session.rb:505)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
RUBY.converting_rjc_exceptions_to_ruby(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare/session.rb:467)
RUBY.new_connection_impl(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare/session.rb:500)
RUBY.initialize(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare/session.rb:136)
RUBY.connect(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare/session.rb:109)
RUBY.connect(/opt/logstash/vendor/bundle/jruby/1.9/gems/march_hare-2.11.0-java/lib/march_hare.rb:20)
RUBY.connect(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-mixin-rabbitmq_connection-2.2.0-java/lib/logstash/plugin_mixins/rabbitmq_connection.rb:137)
RUBY.connect!(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-mixin-rabbitmq_connection-2.2.0-java/lib/logstash/plugin_mixins/rabbitmq_connection.rb:94)
RUBY.register(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-output-rabbitmq-3.0.6-java/lib/logstash/outputs/rabbitmq.rb:40)
org.jruby.RubyArray.each(org/jruby/RubyArray.java:1613)
RUBY.start_outputs(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:192)
RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/pipeline.rb:102)
RUBY.execute(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/agent.rb:165)
RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/runner.rb:90)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
RUBY.run(/opt/logstash/vendor/bundle/jruby/1.9/gems/logstash-core-2.1.1-java/lib/logstash/runner.rb:95)
org.jruby.RubyProc.call(org/jruby/RubyProc.java:281)
RUBY.initialize(/opt/logstash/vendor/bundle/jruby/1.9/gems/stud-0.0.22/lib/stud/task.rb:24)
java.lang.Thread.run(java/lang/Thread.java:745)
And then logstash "crashes" i.e. stops. Does anyone know if that has to do with the config of logstash or is it a problem in rabbitMQ setup?
Thanks in advance.
Daniel
Found my error the port for rabbitMQ has to be 5672 not 15672.
I tried to connect to a particular RabbitMq machine using following code using MassTransit
var bus = ServiceBusFactory.New(x =>
{
x.UseRabbitMq();
x.ReceiveFrom("rabbitmq://admin:admin#<IP Address>/somequeue");
moreInitialization(x);
});
I have create a user as admin in vhost "\" with administrator privileges.
cmd:> rabbitmqctl list_permissions
Listing permissions in vhost "/" ...
admin .* .* .*
I have removed the "guest" user. Even though I user the below code, I can see in rabbitmq log as below
=ERROR REPORT==== 21-Jun-2015::12:19:22 ===
closing AMQP connection <0.391.0> (192.168.6.1:7000 -> 192.168.6.131:5672):
{handshake_error,starting,0,
{amqp_error,access_refused,
"PLAIN login refused: user 'guest' - invalid credentials",
'connection.start_ok'}}
At application side I am getting the below error
Exception:Thrown: "ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile." (RabbitMQ.Client.Exceptions.AuthenticationFailureException)
A RabbitMQ.Client.Exceptions.AuthenticationFailureException was thrown: "ACCESS_REFUSED - Login was refused using authentication mechanism PLAIN. For details see the broker logfile."
Do I need to change any configuration in rabbitmq.config file? I am using RabbitMq Server version 3.5.3
Thanks for helping
To specify credentials, you need to configure the host explicitly. The # syntax is no longer supported, due to leakage of security credentials.
For example:
var bus = ServiceBusFactory.New(sbc =>
{
sbc.UseRabbitMq(r => r.ConfigureHost(new Uri("rabbitmq://actual-server-name/queue-name"), h =>
{
h.SetUsername("username");
h.SetPassword("password");
}));
sbc.ReceiveFrom("rabbitmq://actual-server-name/queue-name");
});