Calico Felix-Typha connection cancelled on ARM64 EKS - amazon-eks

I'm attempting to get Calico installed on an Graviton EKS cluster using the manifests listed here: https://docs.aws.amazon.com/eks/latest/userguide/calico.html
In order to successfully run on ARM64, I'm using a tigera ImageSet with the sha256 manifest of the master-arm64 tags for calico and tigera containers (when they exist). ref: https://projectcalico.docs.tigera.io/maintenance/image-options/imageset
apiVersion: operator.tigera.io/v1
kind: ImageSet
metadata:
name: calico-master
spec:
images:
- image: "calico/apiserver"
digest: "sha256:1a2bc0bad25eb95e77353d59e6ad9edc9d56aa9caebdcfbd027e8ddb7eb956b1"
- image: "calico/cni"
digest: "sha256:a257ee22e3d9e74d2b4c6362045147002104cea6101d3aaefa74661b91fea89b"
- image: "calico/kube-controllers"
digest: "sha256:fd101df470937e14033f602e5817e31e46933c6088a8bdc6fc80e43a1c9e011b"
- image: "calico/node"
digest: "sha256:8694683b9bd0d13caef2e67f1486ded0e843c810f1eb9d4c021a5ffdedd4af8d"
- image: "calico/typha"
digest: "sha256:174b0c47db4297623500cc044826bc259af28974cf5e0df4f84244e824cfda52"
- image: "calico/pod2daemon-flexvol"
digest: "sha256:a276db19af1cba49b7a032ee259e0e0f198575d8af27c9cadfebfe4d63bf15bf"
- image: "calico/windows-upgrade"
digest: "sha256:0123456789abcdef0123456789abcdef0123456789abcdef0123456789abcdef"
- image: "tigera/operator"
digest: "sha256:01327468115202b72519fbe99344b2cc64cca37302d12840827190d97c2ba9cb"
- image: "tigera/key-cert-provisioner"
digest: "sha256:b8b4f0ae606626e029c77dc1c30199f8484f797d2bc52d8e484efc5b938725ad"
Tigera and Calico seem to launch fine, but my calico-node daemonset remains 0/1 forever due to the Felix-Typha connection:
kubectl -n calico-system get all
The logs for Typha (calico-typha deployment):
2022-03-03 15:27:34.692 [INFO][7] sync_server.go 368: Accepted from 192.168.40.212:49482 port=5473
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 393: New connection connID=0x3e1 port=5473
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 558: Per-connection goroutine started client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 636: Failed to read from client client=192.168.40.212:49482 connID=0x3e1 error=gob: name not registered for interface: "github.com/projectcalico/calico/typha/pkg/syncproto.MsgClientHello" thread="read"
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 629: Read goroutine finished client=192.168.40.212:49482 connID=0x3e1 thread="read"
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 666: Asked to stop by context. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [WARNING][7] sync_server.go 675: Failed to read client hello. client=192.168.40.212:49482 connID=0x3e1 error=context canceled
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 545: Client connection shutting down. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 554: Client connection shut down. client=192.168.40.212:49482 connID=0x3e1
2022-03-03 15:27:34.705 [INFO][7] sync_server.go 421: Connection handler finished error=context canceled
The logs for Felix (calico-node daemonset):
2022-03-03 15:33:40.427 [INFO][13853] status-reporter/startup.go 425: Early log level set to info
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAK8SSERVICENAME=calico-typha
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAK8SNAMESPACE=calico-system
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHAKEYFILE=/felix-certs/key.key
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACERTFILE=/felix-certs/cert.crt
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACAFILE=/typha-ca/caBundle
2022-03-03 15:33:40.428 [INFO][13853] status-reporter/config.go 60: Found FELIX_TYPHACN=typha-server
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/discovery.go 163: Found ready Typha addresses. addrs=[]string{"192.168.193.138:5473", "192.168.36.92:5473"}
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/discovery.go 166: Chose Typha to connect to. choice="192.168.36.92:5473"
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/startsyncerclient.go 56: Connecting to Typha. addr="192.168.36.92:5473"
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 71: requiringTLS=true
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 200: Starting Typha client
2022-03-03 15:33:40.447 [INFO][13853] status-reporter/sync_client.go 71: requiringTLS=true
2022-03-03 15:33:40.448 [INFO][13853] status-reporter/tlsutils.go 39: Make certificate verifier requiredCN="typha-server" requiredURISAN="" roots=&x509.CertPool{byName:map[string][]int{"0,1*0(\x06\x03U\x04\x03\f!tigera-operator-signer#1646270114":[]int{0}}, lazyCerts:[]x509.lazyCert{x509.lazyCert{rawSubject:[]uint8{0x30, 0x2c, 0x31, 0x2a, 0x30, 0x28, 0x6, 0x3, 0x55, 0x4, 0x3, 0xc, 0x21, 0x74, 0x69, 0x67, 0x65, 0x72, 0x61, 0x2d, 0x6f, 0x70, 0x65, 0x72, 0x61, 0x74, 0x6f, 0x72, 0x2d, 0x73, 0x69, 0x67, 0x6e, 0x65, 0x72, 0x40, 0x31, 0x36, 0x34, 0x36, 0x32, 0x37, 0x30, 0x31, 0x31, 0x34}, getCert:(func() (*x509.Certificate, error))(0x6d99f0)}}, haveSum:map[x509.sum224]bool{x509.sum224{0xc0, 0x54, 0x82, 0x63, 0xb1, 0xf5, 0xe0, 0xda, 0x83, 0x69, 0x3f, 0x40, 0x66, 0xf7, 0x5a, 0x72, 0x3a, 0x4e, 0x4a, 0xe6, 0x1a, 0xfe, 0xb0, 0xa5, 0x5d, 0xd1, 0x2e, 0xdf}:true}}
2022-03-03 15:33:40.448 [INFO][13853] status-reporter/sync_client.go 252: Connecting to Typha. address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.455 [INFO][13853] status-reporter/tlsutils.go 46: Verify certificate chain signing address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.461 [INFO][13853] status-reporter/sync_client.go 267: Connected to Typha. address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.461 [INFO][13853] status-reporter/sync_client.go 301: Started Typha client main loop address="192.168.36.92:5473" connID=0x0 type="node-status"
2022-03-03 15:33:40.462 [ERROR][13853] status-reporter/sync_client.go 293: Failed to read from server address="192.168.36.92:5473" connID=0x0 error=EOF type="node-status"
2022-03-03 15:33:40.462 [INFO][13853] status-reporter/sync_client.go 166: Typha client Context asked us to exit connID=0x0 type="node-status"
2022-03-03 15:33:40.462 [FATAL][13853] status-reporter/startsyncerclient.go 77: Connection to Typha failed

Related

rabbitmq-server start failed

Facing issue with rabbitmq server startup. Rabbitmq-server start fails with logs -
noti] <0.147.0> Protocol 'inet_tcp': register/listen error: etimedout
noti] <0.147.0>
erro] <0.144.0> supervisor: {local,net_sup}
erro] <0.144.0> errorContext: start_error
erro] <0.144.0> reason: {'EXIT',nodistribution}
erro] <0.144.0> offender: [{pid,undefined},
erro] <0.144.0> {id,net_kernel},
erro] <0.144.0> {mfargs,{net_kernel,start_link,
erro] <0.144.0> [[rabbit_prelaunch_186103#localhost,
erro] <0.144.0> shortnames],
erro] <0.144.0> false,net_sup_dynamic]}},
erro] <0.144.0> {restart_type,permanent},
erro] <0.144.0> {shutdown,2000},
erro] <0.144.0> {child_type,worker}]
erro] <0.144.0>
BOOT FAILED
[erro] <0.131.0>
[erro] <0.131.0> BOOT FAILED
[erro] <0.131.0> ===========
[erro] <0.131.0> Exception during startup:
[erro] <0.131.0>
[erro] <0.131.0> error:{badmatch,{error,{{shutdown,{failed_to_start_child,net_kernel,{'EXIT',nodistribution}}},{child,undefined,net_sup_dynamic,{erl_distribution,start_link,[[rabbit_prelaunch_186103#localhost,shortnames],false,net_sup_dynamic]},permanent,1000,supervisor,[erl_distribution]}}}}
[erro] <0.131.0>
[erro] <0.131.0> rabbit_prelaunch_dist:duplicate_node_check/1, line 78
[erro] <0.131.0> rabbit_prelaunch_dist:setup/1, line 23
[erro] <0.131.0> rabbit_prelaunch:do_run/0, line 115
[erro] <0.131.0> rabbit_prelaunch:run_prelaunch_first_phase/0, line 32
[erro] <0.131.0> supervisor:do_start_child_i/3, line 385
[erro] <0.131.0> supervisor:do_start_child/2, line 371
[erro] <0.131.0> supervisor:-start_children/2-fun-0-/3, line 355
[erro] <0.131.0> supervisor:children_map/4, line 1171
This issue is on centos and rabbitmq version is 3.9.11 / erlang 23.3.4
Already tried rebooting the system, deleting /var/lib/rabbitmq/mnesia files and restarting rabbitmq (no luck).
Has anyone faced this issue?

Kafka SSL connection failure on handshake

Kafka client cannot connect to server via SSL connection for some reason. It goes through SSL handshake, I can see it in the client trace log, but then occasionally fails with "disconnected" message. Looks like Kafka server closes the connection after receives Client Hello.
It fails only when I try it from the OpenShift environment. The same build, configuration settings and SSL certificates work when I'm trying to connect from my local developer machine. Tools like OffsetExplorer are also able to connect from my local PC.
I'm trying to understand what can be different in terms of SSL between OpenShift and my local PC.
Here is the log I grep by KafkaListenerEndpointContainer#5-0-C-1 thread:
Line 93734: 2022-05-24 11:10:26,409 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient::leastLoadedNode:722] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Found least loaded node 10.53.15.97:9093 (id: -1 rack: null) with no active connection
Line 93735: 2022-05-24 11:10:26,409 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [DEBUG] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater::maybeUpdate:1176] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Initialize connection to node 10.53.15.97:9093 (id: -1 rack: null) for sending metadata request
Line 93736: 2022-05-24 11:10:26,409 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [DEBUG] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient::initiateConnect:1005] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Initiating connection to node 10.53.15.97:9093 (id: -1 rack: null) using address /10.53.15.97
Line 93736: 2022-05-24 11:10:26,409 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [DEBUG] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient::initiateConnect:1005] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Initiating connection to node 10.53.15.97:9093 (id: -1 rack: null) using address /10.53.15.97
Line 94113: 2022-05-24 11:10:27,386 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::doHandshake:339] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake NEED_WRAP channelId -1, appReadBuffer pos 0, netReadBuffer pos 0, netWriteBuffer pos 0
Line 94115: 2022-05-24 11:10:27,386 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeWrap:472] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeWrap -1
Line 94117: 2022-05-24 11:10:27,386 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::doHandshake:356] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake NEED_WRAP channelId -1, handshakeResult Status = OK HandshakeStatus = NEED_UNWRAP
Line 94122: 2022-05-24 11:10:27,387 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::doHandshake:365] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake NEED_UNWRAP channelId -1, appReadBuffer pos 0, netReadBuffer pos 0, netWriteBuffer pos 307
Line 94124: 2022-05-24 11:10:27,387 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:499] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap -1
Line 94126: 2022-05-24 11:10:27,387 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:519] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap: handshakeStatus NEED_UNWRAP status BUFFER_UNDERFLOW
Line 94128: 2022-05-24 11:10:27,387 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::doHandshake:387] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake NEED_UNWRAP channelId -1, handshakeResult Status = BUFFER_UNDERFLOW HandshakeStatus = NEED_UNWRAP
Line 94134: 2022-05-24 11:10:27,387 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient::leastLoadedNode:719] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Found least loaded connecting node 10.53.15.97:9093 (id: -1 rack: null)
Line 94137: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::doHandshake:365] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake NEED_UNWRAP channelId -1, appReadBuffer pos 0, netReadBuffer pos 0, netWriteBuffer pos 307
Line 94139: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:499] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap -1
Line 94141: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:519] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap: handshakeStatus NEED_UNWRAP status BUFFER_UNDERFLOW
Line 94143: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:499] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap -1
Line 94145: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [TRACE] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::handshakeUnwrap:519] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] SSLHandshake handshakeUnwrap: handshakeStatus NEED_UNWRAP status BUFFER_UNDERFLOW
Line 94148: 2022-05-24 11:10:27,388 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [DEBUG] (org.apache.kafka.common.network.Selector) [org.apache.kafka.common.network.Selector::pollSelectionKeys:606] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Connection with /10.53.15.97 disconnected
Line 94178: 2022-05-24 11:10:27,389 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [DEBUG] (org.apache.kafka.common.network.SslTransportLayer) [org.apache.kafka.common.network.SslTransportLayer::close:198] mdc:()| [SslTransportLayer channelId=-1 key=channel=java.nio.channels.SocketChannel[connection-pending remote=/10.53.15.97:9093], selector=sun.nio.ch.EPollSelectorImpl#41c9572a, interestOps=8, readyOps=0] Failed to send SSL Close message
Line 94208: 2022-05-24 11:10:27,390 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [WARN] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient::processDisconnection:775] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Connection to node -1 (/10.53.15.97:9093) terminated during authentication. This may happen due to any of the following reasons: (1) Authentication failed due to invalid credentials with brokers older than 1.0.0, (2) Firewall blocking Kafka TLS traffic (eg it may only allow HTTPS traffic), (3) Transient network issue.
Line 94210: 2022-05-24 11:10:27,390 [org.springframework.kafka.KafkaListenerEndpointContainer#5-0-C-1] [WARN] (org.apache.kafka.clients.NetworkClient) [org.apache.kafka.clients.NetworkClient$DefaultMetadataUpdater::handleServerDisconnect:1079] mdc:()| [Consumer clientId=ADCAB-0, groupId=ADCAB_GROUP] Bootstrap broker 10.53.15.97:9093 (id: -1 rack: null) disconnected
And here is the listener configuration:
2022-05-24 11:10:23,199 [main] [INFO] (org.apache.kafka.clients.consumer.ConsumerConfig) [org.apache.kafka.common.config.AbstractConfig::logAll:361] mdc:()| ConsumerConfig values:
allow.auto.create.topics = true
auto.commit.interval.ms = 5000
auto.offset.reset = latest
bootstrap.servers = [10.53.15.97:9093, 10.53.15.96:9093]
check.crcs = true
client.dns.lookup = use_all_dns_ips
client.id = ADCAB-0
client.rack =
connections.max.idle.ms = 540000
default.api.timeout.ms = 60000
enable.auto.commit = false
exclude.internal.topics = true
fetch.max.bytes = 52428800
fetch.max.wait.ms = 500
fetch.min.bytes = 1
group.id = ADCAB_GROUP
group.instance.id = null
heartbeat.interval.ms = 3000
interceptor.classes = []
internal.leave.group.on.close = true
internal.throw.on.fetch.stable.offset.unsupported = false
isolation.level = read_uncommitted
key.deserializer = class org.apache.kafka.common.serialization.StringDeserializer
max.partition.fetch.bytes = 1048576
max.poll.interval.ms = 300000
max.poll.records = 500
metadata.max.age.ms = 300000
metric.reporters = []
metrics.num.samples = 2
metrics.recording.level = INFO
metrics.sample.window.ms = 30000
partition.assignment.strategy = [class org.apache.kafka.clients.consumer.RangeAssignor]
receive.buffer.bytes = 65536
reconnect.backoff.max.ms = 1000
reconnect.backoff.ms = 50
request.timeout.ms = 30000
retry.backoff.ms = 100
sasl.client.callback.handler.class = null
sasl.jaas.config = null
sasl.kerberos.kinit.cmd = /usr/bin/kinit
sasl.kerberos.min.time.before.relogin = 60000
sasl.kerberos.service.name = null
sasl.kerberos.ticket.renew.jitter = 0.05
sasl.kerberos.ticket.renew.window.factor = 0.8
sasl.login.callback.handler.class = null
sasl.login.class = null
sasl.login.refresh.buffer.seconds = 300
sasl.login.refresh.min.period.seconds = 60
sasl.login.refresh.window.factor = 0.8
sasl.login.refresh.window.jitter = 0.05
sasl.mechanism = GSSAPI
security.protocol = SSL
security.providers = null
send.buffer.bytes = 131072
session.timeout.ms = 10000
socket.connection.setup.timeout.max.ms = 127000
socket.connection.setup.timeout.ms = 10000
ssl.cipher.suites = null
ssl.enabled.protocols = [TLSv1.2]
ssl.endpoint.identification.algorithm =
ssl.engine.factory.class = null
ssl.key.password = [hidden]
ssl.keymanager.algorithm = SunX509
ssl.keystore.certificate.chain = null
ssl.keystore.key = null
ssl.keystore.location = /opt/keystore/kafka/sbercms/kafka_keystore.jks
ssl.keystore.password = [hidden]
ssl.keystore.type = JKS
ssl.protocol = TLS
ssl.provider = null
ssl.secure.random.implementation = null
ssl.trustmanager.algorithm = PKIX
ssl.truststore.certificates = null
ssl.truststore.location = /opt/keystore/kafka/sbercms/kafka_truststore.jks
ssl.truststore.password = [hidden]
ssl.truststore.type = JKS
value.deserializer = class ru.sberbank.pprb.sbbol.adcab.utils.kafka.LoggingJsonDeserializer
Sorry, there was an environment problem with the SSL certificates. DevOps managed to resolve it.

QEMU how-to allocate specific IRQ number for PCI device?

I'm running qemu-system-x86_64 with my new pci device. And i want to use IRQ 17 (Since driver from kernel listen for IRQ 17). But my PCI device take IRQ 10 or 11. base on interrupt_pin(A,B,C,E).
Then i want to send irq to kernel module by qemu_irq_pulse.
This is how i allocate irq:
pci_config_set_interrupt_pin(pci_dev->config, 1);
d->irq = pci_allocate_irq(pci_dev)
root#hostname:~# cat /proc/interrupts
CPU0
0: 48 IO-APIC 2-edge timer
1: 9 IO-APIC 1-edge i8042
4: 1440 IO-APIC 4-edge ttyS0
8: 1 IO-APIC 8-edge rtc0
9: 0 IO-APIC 9-fasteoi acpi
12: 125 IO-APIC 12-edge i8042
24: 773 PCI-MSI 512000-edge ahci[0000:00:1f.2]
25: 355 PCI-MSI 32768-edge eth0-rx-0
26: 160 PCI-MSI 32769-edge eth0-tx-0
27: 1 PCI-MSI 32770-edge eth0
lspci -nk -vv:
00:1f.3 0880: 10de:0101 (rev 01)
Subsystem: 1af4:1100
Control: I/O+ Mem+ BusMaster- SpecCycle- MemWINV- VGASnoop- ParErr- Stepping- SERR+ FastB2B- DisINTx-
Status: Cap+ 66MHz- UDF- FastB2B- ParErr- DEVSEL=fast >TAbort- <TAbort- <MAbort- >SERR- <PERR- INTx-
Interrupt: pin A routed to IRQ 10
Capabilities: [80] MSI: Enable- Count=1/1 Maskable- 64bit+
Address: 0000000000000000 Data: 0000
Kernel driver failed to request irq since it trying to attach to IRQ 17. I don't want to change kernel side.
This is what i want to see:
root#hostname:~# cat /proc/interrupts
CPU0
0: 2213 IO-APIC
17: 0 IO-APIC 17-fasteoi some_kernel_driver_name
How to allocate interrupt 17 for PCI device in qemu?
Im not sure it is correct answer but for me it helps:
Add to ACPI:
irqs = 17;
aml_append(crs, aml_interrupt(AML_CONSUMER, AML_EDGE,
AML_ACTIVE_HIGH, AML_SHARED,
&irqs, 1));
Also PCI interrupt number looks like somehow depends on PCI vendor_id and device_id.

java 8 soap client Received fatal alert: handshake_failure

I have a the following exception :
com.sun.xml.internal.ws.client.ClientTransportException: HTTP
transport error: javax.net.ssl.SSLHandshakeException: Received fatal
alert: handshake_failure
when I try to send a soap request to a customer's web service
I use jre1.8.0_66 and I get UnlimitedJCEPolicyJDK8 jars in "Java\jre1.8.0_66\lib\security" folder
and I get Cipher.getMaxAllowedKeyLength("AES") = 2147483647
and for some reason I can't communicate with the customer to get the protocol in use or the cipher suite in the server side.
and here my javax.net.debug related logs :
Allow unsafe renegotiation: false
Allow legacy hello messages: true
Is initial handshake: true
Is secure renegotiation: false
http-nio-9700-exec-1, setSoTimeout(0) called
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_256_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_256_CBC_SHA256 for TLSv1.1
%% No cached client session
*** ClientHello, TLSv1.2
RandomCookie: GMT: 1515623991 bytes = { 212, 12, 195, 65, 98, 206, 121, 198, 232, 203, 220, 162, 207, 122, 217, 87, 121, 168, 220, 246, 60, 50, 9, 61, 214, 181, 16, 190 }
Session ID: {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA384, TLS_RSA_WITH_AES_256_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA384, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA384, TLS_DHE_RSA_WITH_AES_256_CBC_SHA256, TLS_DHE_DSS_WITH_AES_256_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA, TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_256_CBC_SHA, TLS_ECDH_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_RSA_WITH_AES_256_CBC_SHA, TLS_DHE_DSS_WITH_AES_256_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, TLS_RSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_ECDSA_WITH_AES_256_GCM_SHA384, TLS_ECDH_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_RSA_WITH_AES_256_GCM_SHA384, TLS_DHE_DSS_WITH_AES_256_GCM_SHA384, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
***
[write] MD5 and SHA1 hashes: len = 237
http-nio-9700-exec-1, WRITE: TLSv1.2 Handshake, length = 237
[Raw write]: length = 242
[Raw read]: length = 5
0000: 15 03 01 00 02 .....
[Raw read]: length = 2
0000: 02 28 .(
http-nio-9700-exec-1, READ: TLSv1 Alert, length = 2
http-nio-9700-exec-1, RECV TLSv1.2 ALERT: fatal, handshake_failure
http-nio-9700-exec-1, called closeSocket()
http-nio-9700-exec-1, handling exception: javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
http-nio-9700-exec-1, called close()
http-nio-9700-exec-1, called closeInternal(true)
Try adding these parameters to your project configuration:
-Dhttps.cipherSuites=SSL_RSA_WITH_RC4_128_MD5
and comment the parameter for disabled algorithms in the java.security file in the jre folder:
jdk.tls.disabledAlgorithms=SSLv3, RC4, MD5withRSA, DH keySize < 768
Can you check if this (Diagnosing TLS, SSL and HTTPS from blogs.oracle.com) helps you?

Tomcat: javax.net.ssl.SSLHandshakeException: no cipher suites in common

I'm trying to setup a remote tomcat server for deployment in IntelliJ.
For some reason the "handshake" fails.
11:44:28 Error running VPS-Tomcat
Unable to connect to the 185.80.128.231:1099, reason:
java.rmi.ConnectIOException: error during JRMP connection establishment; nested exception is:
javax.net.ssl.SSLHandshakeException: Received fatal alert: handshake_failure
I added some debug options to tomcat startup:
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 for TLSv1
Ignoring unsupported cipher suite: TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_RSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_RSA_WITH_AES_128_CBC_SHA256 for TLSv1.1
Ignoring unsupported cipher suite: TLS_DHE_DSS_WITH_AES_128_CBC_SHA256 for TLSv1.1
RMI TCP Connection(16)-78.60.67.248, READ: TLSv1.2 Handshake, length = 207
*** ClientHello, TLSv1.2
RandomCookie: GMT: 1431613995 bytes = { 210, 37, 184, 64, 38, 79, 199, 129, 139, 3, 89, 15, 7, 99, 193, 123, 94, 24, 149, 84, 76, 24, 210, 199, 14, 10, 32, 220 }
Session ID: {}
Cipher Suites: [TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA256, TLS_RSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA256, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_RSA_WITH_AES_128_CBC_SHA256, TLS_DHE_DSS_WITH_AES_128_CBC_SHA256, TLS_ECDHE_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA, TLS_RSA_WITH_AES_128_CBC_SHA, TLS_ECDH_ECDSA_WITH_AES_128_CBC_SHA, TLS_ECDH_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_RSA_WITH_AES_128_CBC_SHA, TLS_DHE_DSS_WITH_AES_128_CBC_SHA, TLS_ECDHE_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDHE_RSA_WITH_AES_128_GCM_SHA256, TLS_RSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_ECDSA_WITH_AES_128_GCM_SHA256, TLS_ECDH_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_RSA_WITH_AES_128_GCM_SHA256, TLS_DHE_DSS_WITH_AES_128_GCM_SHA256, TLS_ECDHE_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_RSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_ECDSA_WITH_3DES_EDE_CBC_SHA, TLS_ECDH_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_RSA_WITH_3DES_EDE_CBC_SHA, SSL_DHE_DSS_WITH_3DES_EDE_CBC_SHA, TLS_ECDHE_ECDSA_WITH_RC4_128_SHA, TLS_ECDHE_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_SHA, TLS_ECDH_ECDSA_WITH_RC4_128_SHA, TLS_ECDH_RSA_WITH_RC4_128_SHA, SSL_RSA_WITH_RC4_128_MD5, TLS_EMPTY_RENEGOTIATION_INFO_SCSV]
Compression Methods: { 0 }
Extension elliptic_curves, curve names: {secp256r1, sect163k1, sect163r2, secp192r1, secp224r1, sect233k1, sect233r1, sect283k1, sect283r1, secp384r1, sect409k1, sect409r1, secp521r1, sect571k1, sect571r1, secp160k1, secp160r1, secp160r2, sect163r1, secp192k1, sect193r1, sect193r2, secp224k1, sect239k1, secp256k1}
Extension ec_point_formats, formats: [uncompressed]
Extension signature_algorithms, signature_algorithms: SHA512withECDSA, SHA512withRSA, SHA384withECDSA, SHA384withRSA, SHA256withECDSA, SHA256withRSA, SHA224withECDSA, SHA224withRSA, SHA1withECDSA, SHA1withRSA, SHA1withDSA, MD5withRSA
%% Initialized: [Session-14, SSL_NULL_WITH_NULL_NULL]
%% Invalidated: [Session-14, SSL_NULL_WITH_NULL_NULL]
RMI TCP Connection(16)-78.60.67.248, SEND TLSv1.2 ALERT: fatal, description = handshake_failure
RMI TCP Connection(16)-78.60.67.248, WRITE: TLSv1.2 Alert, length = 2
RMI TCP Connection(16)-78.60.67.248, called closeSocket()
RMI TCP Connection(16)-78.60.67.248, handling exception: javax.net.ssl.SSLHandshakeException: no cipher suites in common
RMI TCP Connection(16)-78.60.67.248, called close()
RMI TCP Connection(16)-78.60.67.248, called closeInternal(true)
I don't really understand it but I assume that one end is using TLS1.2 and the other TLS1.1 although I'm not even sure which is which.
So how can I configure them to both use the same version? Or make tomcat support those cipher suites?
Or does the problem lie elsewhere?