Elastic Search Docker search service error socket hang up - asp.net-core

I have my docker-compose.override.yml set up below in Visual Studio 2022
elasticsearch:
container_name: elasticsearch
environment:
- "ES_JAVA_OPTS=-Xms512m -Xmx512m"
- discovery.type=single-node
restart: always
ports:
- "9200:9200"
- "9300:9300"
networks:
elastic:
kibana:
container_name: kibana
restart: always
environment:
- ELASTICSEARCH_HOSTS=http://elasticsearch:9200
depends_on:
- elasticsearch
ports:
- "5601:5601"
networks:
elastic:
networks:
elastic:
driver: bridge
Every time I try to configure kibana, after supplying a token, I get the following error from the kibana container w/out kibana getting fully configured.
2023-02-06 21:40:38 [2023-02-07T04:40:38.210+00:00][WARN ][plugins.alerting] APIs are disabled because the Encrypted Saved Objects plugin is missing encryption key. Please set xpack.encryptedSavedObjects.encryptionKey in the kibana.yml or use the bin/kibana-encryption-keys command.
2023-02-06 21:40:38 [2023-02-07T04:40:38.366+00:00][INFO ][plugins.ruleRegistry] Installing common resources shared between all indices
2023-02-06 21:40:38 [2023-02-07T04:40:38.508+00:00][INFO ][plugins.cloudSecurityPosture] Registered task successfully [Task: cloud_security_posture-stats_task]
2023-02-06 21:40:39 [2023-02-07T04:40:39.605+00:00][INFO ][plugins.screenshotting.config] Chromium sandbox provides an additional layer of protection, and is supported for Linux Ubuntu 20.04 OS. Automatically enabling Chromium sandbox.
2023-02-06 21:40:39 [2023-02-07T04:40:39.741+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. socket hang up - Local: 192.168.128.3:60856, Remote: 192.168.128.2:9200
2023-02-06 21:40:40 [2023-02-07T04:40:40.773+00:00][INFO ][plugins.screenshotting.chromium] Browser executable: /usr/share/kibana/x-pack/plugins/screenshotting/chromium/headless_shell-linux_x64/headless_shell
2023-02-06 21:40:42 [2023-02-07T04:40:42.194+00:00][ERROR][elasticsearch-service] Unable to retrieve version information from Elasticsearch nodes. socket hang up - Local: 192.168.128.3:44378, Remote: 192.168.128.2:9200
I am a bit at a loss as to why I can't get it configured. Any help appreciated.

Related

Mailbox provisioning from OpenLDAP/LAM with mailcow's DOVECOT_MASTER_USER

TLDR; The issue is that LAM cannot successfully authenticate to mailcow (Dovecot) with the DOVECOT_MASTER_USER and DOVECOT_MASTER_PASSWORD setup, the connection is aborted and hence no mailbox was provisioned.
So, I am trying to connect a directory service with mailcow, in order to provision and manage mailboxes for users and groups from LDAP Account Manager (LAM).
The directory service is comprised of bitnamit/OpenLDAP image with LDAP Account Manager (LAM) as Directory Management frontend run as a docker-compose stack behind Traefik as Reverse Proxy just as mailcow.
version: '3.5'
volumes:
lam-app:
lam-data:
openldap-data:
networks:
openldap-net:
driver: bridge
web:
external: true
services:
ldap-account-manager:
image: ldapaccountmanager/lam:latest
restart: unless-stopped
volumes:
- lam-data:/var/lib/ldap-account-manager/
- lam-app:/etc/ldap-account-manager
environment:
# - LAM_PASSWORD=${LAM_PASSWORD}
# - LAM_LANG=${LAM_LANG}
# - LDAP_SERVER=${LDAP_SERVER}
# - LDAP_DOMAIN=${LDAP_DOMAIN}
# - LDAP_BASE_DN=${LDAP_BASE_DN}
# - ADMIN_USER=cn=${LDAP_ADMIN_NAME},${LDAP_BASE_DN}
- LAM_PASSWORD=${LAM_PASSWORD}
- LAM_LANG=${LAM_LANG}
- LDAP_SERVER=${LDAP_SERVER}
- LDAP_DOMAIN=${LDAP_DOMAIN}
- LDAP_BASE_DN=${LDAP_BASE_DN}
- ADMIN_USER=cn=admin,${LDAP_BASE_DN}
# - DEBUG=true
hostname: directory
domainname: "${LDAP_DOMAIN}"
networks:
- openldap-net
- web
labels:
- traefik.enable=true
- traefik.docker.network=web
- traefik.http.routers.openldap-lam.rule=Host(`directory.${LDAP_DOMAIN}`)
- traefik.http.routers.openldap-lam.entrypoints=websecure
- traefik.http.routers.openldap-lam.tls=true
- traefik.http.routers.openldap-lam.tls.certresolver=myresolver
- traefik.http.routers.openldap-lam.service=openldap-lam-svc
- traefik.http.services.openldap-lam-svc.loadbalancer.server.port=80
- traefik.http.routers.openldap-lam.middlewares=ldap-auth
- traefik.http.middlewares.ldap-auth.basicauth.users=adam:$$2y$$05$$aio7rx3j9w87ky2uw9r
certdumper:
image: humenius/traefik-certs-dumper
container_name: ldap_certdumper
network_mode: none
# env_file:
# - ldap.env
volumes:
# mount the folder which contains Traefik's `acme.json' file
- ../../proxy/letsencrypt/:/traefik:ro
- ./ldap/certs/:/output:rw
environment:
- DOMAIN=ldap.${LDAP_DOMAIN}
# Bitnami OpenLDAP container (actively maintained)
ldap:
image: bitnami/openldap:latest
ports:
- '389:1389' # Expose port 389
# - '1636:1636'
networks:
- openldap-net
- web
volumes:
- openldap-data:/bitnami/openldap
- ./ldap/schemas:/schemas # Additional LDIF schemas
labels:
- traefik.enable=true
- traefik.docker.network=web
- traefik.http.routers.ldap.rule=Host(`ldap.${LDAP_DOMAIN}`)
- traefik.http.routers.ldap.tls=true
- traefik.http.routers.ldap.tls.certresolver=myresolver
- traefik.http.routers.ldap.entrypoints=ldap
- traefik.http.routers.ldap.service=ldap-svc
- traefik.http.services.ldap-svc.loadbalancer.server.port=1389
environment:
- LDAP_ADMIN_USERNAME=${LDAP_ADMIN_NAME}
- LDAP_ADMIN_PASSWORD=${LDAP_ADMIN_PASSWORD}
- LDAP_ROOT=${LDAP_BASE_DN}
- LDAP_SKIP_DEFAULT_TREE=${LDAP_SKIP_DEFAULT_TREE}
- LDAP_GROUP=${LDAP_GROUP}
- LDAP_USER_DC=${LDAP_USER_DC}
- LDAP_CONFIG_ADMIN_ENABLED=${LDAP_CONFIG_ADMIN_ENABLED}
- LDAP_CONFIG_ADMIN_USERNAME=${LDAP_CONFIG_ADMIN_NAME}
- LDAP_CONFIG_ADMIN_PASSWORD=${LDAP_CONFIG_ADMIN_PASSWORD}
- LDAP_LOGLEVEL=${LDAP_LOGLEVEL}
- LDAP_PORT_NUMBER=${LDAP_PORT}
- LDAP_USERS=${LDAP_USERS}
- LDAP_PASSWORDS=${LDAP_PASSWORDS}
As LAM documentation states there are no additional schemas needed on the LDAP side, in order to setup and use IMAP mailboxes. But as can be seen I mount several LDIF schemas to extend the basic LDAP functionality. Namely:
asterisk.ldif: Software-PBX
dhcp.schema: DHCP from LDAP
freeradius.ldif: RADIUS from LDAP
ldapns.ldif: Hosts and Allowed Services Management
misc.ldif: E-Mail aliases among other things.
pyroka: printer access and management
IMAP settings in LAM are set as follows:
On the mailcow side, I adjusted mailcow.conf and set DOVECOT_MASTER_USER and DOVECOT_MASTER_PASSWORD. On restart I can find the new user and password created in /etc/dovecot/dovecot-master.userdb and /etc/dovecot/dovecot-master.passdb.
So far so good; LAM connect to Dovecot service of the respective mailcow instance and tries to authenticate with the username / password setup for the DOVECOT_MASTER_USER. But authentication fails with the following logs:
mailcowdockerized-dovecot-mailcow-1 | Feb 4 23:53:53 374f0ac661be dovecot: imap-login: Disconnected: Aborted login by logging out (auth failed, 1 attempts in 2 secs): user=<dovecot-master#mailcow.local>, method=PLAIN, rip=fd4d:6169:6c63:6f77::1, lip=fd4d:6169:6c63:6f77::f, TLS, TLSv1.2 with cipher ECDHE-RSA-AES256-GCM-SHA384 (256/256 bits)
Could it be due to TLS and certificates? OpenLDAP has TLS turned off internally. TLS termination is done with Traefik, which obtains and manages certificates via Letencrypt. Could this setup not meet Mailcow’s encryption requirements for successful authentication?
Would be grateful for hints and help 🙂
Luke

Dapr Service Invocation Connection Refused issue

I have a ASP.Net core Razor app that calls an api called TokenAPI, as well as another API called DevOps that calls the Token API, using the Service Invocation block. All of these are hosted in DAPR.
The web app and 2 APIs are hosted locally with Docker Compose. The web app can sucessfully call the Token API using the service invocation block, but the DevOps API fails with
"Exception":"System.Net.Http.HttpRequestException: Connection refused (127.0.0.1:3500)"
I am at a loss as to why this is not working and have been comparing the configuration in my docker-compose file, but cannot figure out why the web-to-api works, whilst the api-to-api fails. Both the Web App and DevOps api uses the same code and config used to call the Token API when creating calling CreateInvokeHttpClient during app startup. So, with code being te same, the only place I can think of, must be the docker-compose:
cas.tokens.api:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80;https://+:443
ports:
- "5105:80"
- "5115:443"
- "50005:50001"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
cas.devops.api:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80;https://+:443
ports:
- "5106:80"
- "5116:443"
- "50006:50001"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
cas.web:
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=https://+:443;http://+:80
ports:
- "5111:443"
volumes:
- ${APPDATA}/Microsoft/UserSecrets:/root/.microsoft/usersecrets:ro
- ${APPDATA}/ASP.NET/Https:/root/.aspnet/https:ro
networks:
- mynetwork
castokensapi-dapr:
command: ["./daprd",
"-app-id", "tokensapi",
"-app-port", "80", --enable-api-logging,
"-dapr-http-port", "3605",
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
casdevopsapi-dapr:
command: ["./daprd",
"-app-id", "devopsapi",
"-app-port", "80", --enable-api-logging,
"-dapr-http-port", "3606",
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
casweb-dapr:
command: [
"./daprd",
"-app-id", "casweb", --enable-api-logging,
"-placement-host-address", "dapr-placement:50000",
"-components-path", "/components",
"-config", "/configuration/config.yaml"
]
volumes:
- "./dapr/components/:/components"
- "./dapr/configuration/:/configuration"
I am confused as to why DAPR resolves the endpoint to be 127.0.0.1:3500 instead of 127.0.0.1:3605, as per the config?
Below is a screenshot of what the app-id resolves for both service invocations from web-to-api to api-to-api.
The issue seems to be that the service location is either not working or, there is something more fundamental that I misunderstand.
Any help is much appreciated
[UPDATE]
I managed to get this to work, but not entirely sure why yet lol. Removing the "-dapr-http-port" fixes this issue, but I would still like to know why.
Will post here when I know more.
-Mike

How do I know if a plugin actually runs or not?

I'm trying out traefik and developing a dummy plugin right now. After countless of errors, I finally get rid all of the errors and don't get any error anymore.
But, the plugin doesn't work as intended. It doesn't throw errors, but it doesn't seem to work either. Is there any way to confirm that the Plugin actually runs? By "running", I mean that I configure it properly, but the function just doesn't return what I want.
This is the output after running the docker compose.
echo-server | Echo server listening on port 8080.
traefik-proxy | time="2022-03-21T07:58:28Z" level=info msg="Configuration loaded from flags."
That's just it. No errors, no exit. And it throws a GET log after refreshing the web page, so I assume there are no errors blocking the code. But I'm still not what's wrong. Is it the plugin code or the configuration?
If this is necessary, this is some of my codes:
# docker-compose.yml
version: "3.3"
networks:
traefik-proxy:
volumes:
traefik-proxy:
services:
traefik-proxy:
image: "traefik:latest"
container_name: "traefik-proxy"
networks:
- traefik-proxy
command:
# - "--log.level=DEBUG"
- "--api.insecure=true"
- "--providers.docker=true"
- "--providers.docker.exposedbydefault=false"
- "--entrypoints.web.address=:80"
- "--entrypoints.websecure.address=:443"
- "--experimental.localPlugins.traefik-denyuseragent.modulename=github.com/xxx/denyuseragent"
ports:
- "80:80"
- "8080:8080"
- "443:443"
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
- ./plugins-local/:/plugins-local/
echo-server:
image: "xxx/echo-server"
container_name: "echo-server"
networks:
- traefik-proxy
labels:
- "traefik.enable=true"
- "traefik.http.routers.echoserver.rule=Host(`echoserver.localhost`)"
- "traefik.http.routers.echoserver.entrypoints=web"
- "traefik.http.routers.echoserver.middlewares=traefik-denyuseragent"
# .traefik.yml
displayName: Plugin
type: middleware
import: github.com/xxx/denyuseragent
summary: 'Example'
testData:
userAgent:
- Firefox
- Mozilla/5.0
# traefik.yml
experimental:
localPlugins:
traefik-denyuseragent:
modulename: "github.com/xxx/denyuseragent"
# config.yml
http:
routers:
my-router:
rule: host(`echoserver.localhost`)
service: service-echoserver
entryPoints:
- web
middlewares:
- traefik-denyuseragent
services:
service-echoserver:
loadBalancer:
servers:
- url: http://127.0.0.1:5000
middlewares:
traefik-denyuseragent:
plugin:
traefik-denyuseragent:
userAgent:
- Mozilla/5.0
Anyone can confirm if it could work properly or not? Is there some kinds of code I can run to make sure if the Plugin is configured properly or not? Because it'd be great if it is so I can move on to another task, I've spent days just try to configure it.
I just wanna know if the problem is on the plugin or the configuration.

GCS Connector fails when enabling SASL, SSL, or SASL_SSL

I have been able to successfully connect the GCS connector without SASL or SSL enabled. When I enable SASL and SSL; connect-standalone does not seem to be able to communicate with the brokers.
The problem appears to be with the gcs-sink-license-manager. This is what I have found from the logs but they aren't super helpful for me to actually figuring out what the issue is....
LOGS
[2018-12-19 16:29:05,645] INFO [AdminClient clientId=gcs-sink-license-manager] Metadata update failed (org.apache.kafka.clients.admin.internals.AdminMetadataManager:238)
org.apache.kafka.common.errors.TimeoutException: Timed out waiting to send the call.
[2018-12-19 16:29:05,647] ERROR WorkerConnector{id=gcs-sink} Error while starting connector (org.apache.kafka.connect.runtime.WorkerConnector:119)
org.apache.kafka.connect.errors.ConnectException: Timed out while checking for or creating topic(s) '_confluent-command'. This could indicate a connectivity issue, unavailable topic partitions, or if this is your first use of the topic it may have taken too long to create.
at org.apache.kafka.connect.util.TopicAdmin.createTopics(TopicAdmin.java:251)
at io.confluent.license.LicenseStore$1.run(LicenseStore.java:159)
at org.apache.kafka.connect.util.KafkaBasedLog.start(KafkaBasedLog.java:126)
at io.confluent.license.LicenseStore.start(LicenseStore.java:187)
at io.confluent.license.LicenseManager.<init>(LicenseManager.java:42)
at io.confluent.connect.gcs.GcsSinkConnector.checkLicense(GcsSinkConnector.java:80)
at io.confluent.connect.gcs.GcsSinkConnector.start(GcsSinkConnector.java:67)
at org.apache.kafka.connect.runtime.WorkerConnector.doStart(WorkerConnector.java:111)
at org.apache.kafka.connect.runtime.WorkerConnector.start(WorkerConnector.java:136)
at org.apache.kafka.connect.runtime.WorkerConnector.transitionTo(WorkerConnector.java:195)
at org.apache.kafka.connect.runtime.Worker.startConnector(Worker.java:241)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.startConnector(StandaloneHerder.java:297)
at org.apache.kafka.connect.runtime.standalone.StandaloneHerder.putConnectorConfig(StandaloneHerder.java:206)
at org.apache.kafka.connect.cli.ConnectStandalone.main(ConnectStandalone.java:107)
Caused by: org.apache.kafka.common.errors.TimeoutException: Timed out waiting for a node assignment.
[2018-12-19 16:29:05,649] INFO Finished creating connector gcs-sink (org.apache.kafka.connect.runtime.Worker:257)
[2018-12-19 16:29:05,650] INFO Skipping reconfiguration of connector gcs-sink since it is not running (org.apache.kafka.connect.runtime.standalone.StandaloneHerder:329)
[2018-12-19 16:29:05,652] INFO Created connector gcs-sink (org.apache.kafka.connect.cli.ConnectStandalone:104)
Connector Properties
connector.class="io.confluent.connect.gcs.GcsSinkConnector"
storage.class="io.confluent.connect.gcs.storage.GcsStorage"
bootstrap.servers=kafka1:19092
key.converter=org.apache.kafka.connect.json.JsonConverter
value.converter=org.apache.kafka.connect.json.JsonConverter
key.converter.schemas.enable=false
value.converter.schemas.enable=false
security.protocol=SASL_PLAINTEXT
sasl.mechanism=PLAIN
offset.storage.file.filename=/tmp/connect.offsets
offset.flush.interval.ms=10000
plugin.path=/usr/share/java,/usr/share/confluent-hub-components
gcs.sasl.properties
#Connector
format.class=io.confluent.connect.gcs.format.json.JsonFormat
partitioner.class=io.confluent.connect.storage.partitioner.DefaultPartitioner
flush.size=3
# confluent.license=
#GCS
name=gcs-sink
connector.class=io.confluent.connect.gcs.GcsSinkConnector
gcs.bucket.name=kafka-bucket-4c
gcs.part.size=5242880
gcs.credentials.path=/usr/share/assets/gcs-key.json
confluent.topic.bootstrap.servers=kafka1:19092
topics=sandbox
confluent.topic.replication.factor=1
#Storage
storage.class=io.confluent.connect.gcs.storage.GcsStorage
client.id=gcs-standalone-sink
# Sink authentication settings
consumer.log4j.root.loglevel=DEBUG
consumer.bootstrap.servers=kafka1:19092
consumer.sasl.mechanism=PLAIN
consumer.security.protocol=SASL_PLAINTEXT
consumer.ssl.endpoint.identification.algorithm=
Dockerfile
FROM confluentinc/cp-kafka-connect
ADD assets /usr/share/assets
# ENV CONNECT_OPTS "-Djava.security.auth.login.config=/usr/share/assets/kafka_admin_account.conf -Djavax.net.ssl.trustStore=/usr/share/assets/secrets/kafka.client.truststore.jks -Djavax.net.ssl.trustStorePassword=changeit"
ENV KAFKA_OPTS "-Djava.security.auth.login.config=/usr/share/assets/secrets/kafka_admin_account.conf -Djavax.net.debug=all"
ENV CONNECT_OPTS "-Djava.security.auth.login.config=/usr/share/assets/secrets/kafka_admin_account.conf -Djavax.net.debug=all"
COPY assets/secrets/cacerts /usr/lib/jvm/zulu-8-amd64/jre/lib/security/cacerts
CMD ["/bin/bash", "-c", "connect-standalone ${CONNECT_PROPS} ${GCS_PROPS}"]
docker-compose file
kafka1:
image: company-kafka-secure
# build: ./
depends_on:
- zookeeper
ports:
- 19091:19091
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: SASL_PLAINTEXT://kafka1:19092,EXT://localhost:19091
KAFKA_LISTENERS: SASL_PLAINTEXT://:19092,EXT://:19091
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: SASL_PLAINTEXT:SASL_PLAINTEXT,EXT:SASL_PLAINTEXT
KAFKA_SASL_MECHANISM_INTER_BROKER_PROTOCOL: PLAIN
KAFKA_SECURITY_INTER_BROKER_PROTOCOL: SASL_PLAINTEXT
KAFKA_SASL_ENABLED_MECHANISMS: PLAIN
KAFKA_ZOOKEEPER_CONNECTION_TIMEOUT_MS: 6000
ZOOKEEPER_SASL_ENABLED: "false"
KAFKA_AUTHORIZER_CLASS_NAME: com.us.digital.kafka.security.authorization.KafkaAuthorizer
CONFLUENT_METRICS_ENABLE: "false"
volumes:
- ./secrets:/etc/kafka/secrets
networks:
- message_hub
kafka_gcs_connect:
build: ./kafka-connect
ports:
- 28082:28082
depends_on:
- kafka1
- kafka3
- kafka2
- zookeeper
environment:
CONNECT_PROPS: /usr/share/assets/connect-standalone.sasl.properties
CONNECT_REST_PORT: 28082
GCS_PROPS: /usr/share/assets/gcs.sasl.properties
networks:
- message_hub
CONNECT_BOOTSTRAP_SERVERS=kafka1:19092,kafka2:29092,kafka3:39092
CONNECT_CONFLUENT_TOPIC_BOOTSTRAP_SERVERS=kafka1:19092,kafka2:29092,kafka3:39092
CONNECT_CONFLUENT_LICENSE=
CONNECT_KEY_CONVERTER=org.apache.kafka.connect.json.JsonConverter
CONNECT_VALUE_CONVERTER=org.apache.kafka.connect.json.JsonConverter
CONNECT_KEY_CONVERTER_SCHEMAS_ENABLE=false
CONNECT_VALUE_CONVERTER_SCHEMAS_ENABLE=false
CONNECT_CONFIG_STORAGE_TOPIC=connect-config
CONNECT_OFFSET_STORAGE_TOPIC=connect-offsets
CONNECT_STATUS_STORAGE_TOPIC=connect-status
CONNECT_REPLICATION_FACTOR=1
CONNECT_CONFIG_STORAGE_REPLICATION_FACTOR=1
CONNECT_OFFSET_STORAGE_REPLICATION_FACTOR=1
CONNECT_STATUS_STORAGE_REPLICATION_FACTOR=1
CONNECT_SECURITY_PROTOCOL=SASL_PLAINTEXT
CONNECT_SASL_MECHANISM=PLAIN
CONNECT_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=
CONNECT_CONSUMER_BOOTSTRAP_SERVERS=kafka1:19092,kafka2:29092,kafka3:39092
CONNECT_CONSUMER_SECURITY_PROTOCOL=SASL_PLAINTEXT
CONNECT_CONSUMER_SSL_ENDPOINT_IDENTIFICATION_ALGORITHM=
CONNECT_CONSUMER_SASL_MECHANISM=PLAIN
CONNECT_GROUP_ID=gcs-kafka-connector
CONNECT_INTERNAL_KEY_CONVERTER=org.apache.kafka.connect.json.JsonConverter
CONNECT_INTERNAL_VALUE_CONVERTER=org.apache.kafka.connect.json.JsonConverter
CONNECT_REST_PORT=28082
CONNECT_PLUGIN_PATH=/usr/share/java,/usr/share/confluent-hub-components
KAFKA_OPTS=-Djava.security.auth.login.config=/usr/share/assets/kafka_admin_account.conf
Here is all of the properties I found I needed to get SASL working with a gcs connector.

WildFly Swarm apps using an external ActiveMQ broker

I'm having a very hard time to get two WildFly swarm apps (based on 2017.9.5 version) communicate with each other over a standalone ActiveMQ 5.14.3 broker. All done using YAML config as I can't have a main method in my case.
after reading hundreds of outdated examples and inaccurate pages of documentation, I settled with following settings for both producer and consumer apps:
swarm:
messaging-activemq:
servers:
default:
jms-topics:
domain-events: {}
messaging:
remote:
name: remote-mq
host: localhost
port: 61616
jndi-name: java:/jms/remote-mq
remote: true
Now it seems that at least part of the setting is correct as the apps start except for following warning:
2017-09-16 14:20:04,385 WARN [org.jboss.activemq.artemis.wildfly.integration.recovery] (MSC service thread 1-2) AMQ122018: Could not start recovery discovery on XARecoveryConfig [transportConfiguration=[TransportConfiguration(name=, factory=org-apache-activemq-artemis-core-remoting-impl-netty-NettyConnectorFactory) ?port=61616&localAddress=::&host=localhost], discoveryConfiguration=null, username=null, password=****, JNDI_NAME=java:/jms/remote-mq], we will retry every recovery scan until the server is available
Also when producer tries to send messages it just times out and I get following exception (just the last part):
Caused by: javax.jms.JMSException: Failed to create session factory
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:727)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createXAConnection(ActiveMQConnectionFactory.java:304)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createXAConnection(ActiveMQConnectionFactory.java:300)
at org.apache.activemq.artemis.ra.ActiveMQRAManagedConnection.setup(ActiveMQRAManagedConnection.java:785)
... 127 more
Caused by: ActiveMQConnectionTimedOutException[errorType=CONNECTION_TIMEDOUT message=AMQ119013: Timed out waiting to receive cluster topology. Group:null]
at org.apache.activemq.artemis.core.client.impl.ServerLocatorImpl.createSessionFactory(ServerLocatorImpl.java:797)
at org.apache.activemq.artemis.jms.client.ActiveMQConnectionFactory.createConnectionInternal(ActiveMQConnectionFactory.java:724)
... 130 more
I suspect that the problem is ActiveMQ has security turned on, but I found no place to give username and password to swarm config.
The ActiveMQ instance is running using Docker and following compose file:
version: '2'
services:
activemq:
image: webcenter/activemq
environment:
- ACTIVEMQ_NAME=amqp-srv1
- ACTIVEMQ_REMOVE_DEFAULT_ACCOUNT=true
- ACTIVEMQ_ADMIN_LOGIN=admin
- ACTIVEMQ_ADMIN_PASSWORD=your_password
- ACTIVEMQ_WRITE_LOGIN=producer_login
- ACTIVEMQ_WRITE_PASSWORD=producer_password
- ACTIVEMQ_READ_LOGIN=consumer_login
- ACTIVEMQ_READ_PASSWORD=consumer_password
- ACTIVEMQ_JMX_LOGIN=jmx_login
- ACTIVEMQ_JMX_PASSWORD=jmx_password
- ACTIVEMQ_MIN_MEMORY=1024
- ACTIVEMQ_MAX_MEMORY=4096
- ACTIVEMQ_ENABLED_SCHEDULER=true
ports:
- "1883:1883"
- "5672:5672"
- "8161:8161"
- "61616:61616"
- "61613:61613"
- "61614:61614"
any idea what's going wrong?
I had bad times trying to get it working too. The following YML solved my problem:
swarm:
network:
socket-binding-groups:
standard-sockets:
outbound-socket-bindings:
myapp-socket-binding:
remote-host: localhost
remote-port: 61616
messaging-activemq:
servers:
default:
remote-connectors:
myapp-connector:
socket-binding: myapp-socket-binding
pooled-connection-factories:
myAppRemote:
user: username
password: password
connectors:
- myapp-connector
entries:
- 'java:/jms/remote-mq'