Problems setting up SSL locally with Phoenix/Elixir - ssl

I've created a new Phoenix web application on OSX and I'm trying to get SSL working on localhost. To do that, I read and performed the steps of this article. So now I have a server.key, server.crt and server.csr files. The files are not binary and are in readable form. I placed those files in the priv folder as the Phoenix docs suggested.
My config file looks like this:
config :{{name}}, {{name}}.Endpoint,
http: [port: 4000],
debug_errors: true,
code_reloader: true,
cache_static_lookup: false,
check_origin: false,
watchers: [node: ["node_modules/brunch/bin/brunch", "watch", "--stdin"]],
https: [port: 4043,
otp_app: :{{name}},
keyfile: System.get_env("server.key"),
certfile: System.get_env("server.crt"),
# OPTIONAL Key for intermediate certificates
# cacertfile: System.get_env("INTERMEDIATE_CERTFILE_PATH")
]
When I run mix phoenix.server I'm getting the following error:
** (Mix) Could not start application {{name}}: {{name}}.start(:normal, []) returned an error: shutdown: failed to start child: {{name}}.Endpoint
** (EXIT) shutdown: failed to start child: Phoenix.Endpoint.Server
** (EXIT) shutdown: failed to start child: {:ranch_listener_sup, {{name}}.Endpoint.HTTPS}
** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
** (EXIT) an exception was raised:
** (MatchError) no match of right hand side value: {:error, {:options, {:certfile, nil}}}
(ranch) src/ranch_acceptors_sup.erl:30: :ranch_acceptors_sup.init/1
(stdlib) supervisor.erl:243: :supervisor.init/1
(stdlib) gen_server.erl:306: :gen_server.init_it/6
(stdlib) proc_lib.erl:237: :proc_lib.init_p_do_apply/3
What am I doing wrong? I'm a Phoenix newbie and the project I'm working on requires SSL on localhost to prevent cross-domain issues.

It seems phoenix is not able to find your certificates.
To overcome this problem, you can either provide an absolute path or can take advantage of otp_app to use a relative path where phoenix can search for the certificates. If you provide otp_app, phoenix will look in your application root for certificates.
If you want to provide an absolute path you can do something like that:
keyfile: Path.expand("../../../some/path/to/ssl/cer.key", __DIR__),
certfile: Path.expand("../../../some/path/to/ssl/cer.crt", __DIR__)
If you want to take advantage of otp_app, create two env variable say KEY_HOME and CERT_HOME. Go to console and fire these two commands. You should add them to your bashrc file later.
export KEY_HOME=priv/ssl/server.key
export CERT_HOME=priv/ssl/server.crt
you must include priv directory here
Now your config looks like this
https: [port: 443,
otp_app: :hello_phoenix,
keyfile: System.get_env("KEY_HOME"),
certfile: System.get_env("CERT_HOME")
]
Don't forget to copy your files in priv/ssl.

Related

JMeter JMS: Getting Error: Failed to connect via SSL to [ssl://<server-name>:7352]: Received fatal alert: protocol_version

I'm trying to publish message to a Tibco Queue on a SSL Tibco Server through JMeter 5.4.1 using JMS Point-to-Point Logic Controller.
JMS Point To Point Controller Config
But I'm getting the following error message:
2021-06-13 12:25:46,278 ERROR o.a.j.p.j.s.JMSSampler: Not permitted:
Failed to connect to any server at: ssl://[server-name]:7352,
ssl://[server-name]:7352 [Error: Failed to connect via SSL to
[ssl://[server-name]:7352]: Received fatal alert:
protocol_version: url that returned this exception =
SSL://[server-name]:7352 ]
javax.naming.AuthenticationException: Not permitted: Failed to connect
to any server at: ssl://[server-name]:7352,
ssl://[server-name]:7352 [Error: Failed to connect via SSL to
[ssl://[server-name]:7352]: Received fatal alert:
protocol_version: url that returned this exception =
SSL://[server-name] ] at
com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:670)
~[tibjms.jar:8.0.0] at
com.tibco.tibjms.naming.TibjmsContext.lookup(TibjmsContext.java:491)
~[tibjms.jar:8.0.0] at
javax.naming.InitialContext.lookup(InitialContext.java:417)
~[?:1.8.0_291] at
org.apache.jmeter.protocol.jms.sampler.JMSSampler.threadStarted(JMSSampler.java:638)
[ApacheJMeter_jms.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread$ThreadListenerTraverser.addNode(JMeterThread.java:784)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jorphan.collections.HashTree.traverseInto(HashTree.java:993)
[jorphan.jar:5.4.1] at
org.apache.jorphan.collections.HashTree.traverse(HashTree.java:976)
[jorphan.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.threadStarted(JMeterThread.java:752)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.initRun(JMeterThread.java:740)
[ApacheJMeter_core.jar:5.4.1] at
org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:252)
[ApacheJMeter_core.jar:5.4.1]
I tried:
openssl s_client -connect [server-name]:7352
It gave the following output:
SSL-Session:
Protocol : TLSv1.2
Cipher : ECDHE-RSA-AES256-GCM-SHA384
Session-ID:
Session-ID-ctx:
So added the following line in jmeter.properties file.
https.default.protocol=TLSv1.2
Also commented jdk.tls.disabledAlgorithms from java.security file for JDK (I'm using jdk1.8.0_291)
# jdk.tls.disabledAlgorithms=SSLv3, TLSv1, TLSv1.1, RC4, DES, MD5withRSA, \
# DH keySize < 1024, EC keySize < 224, 3DES_EDE_CBC, anon, NULL, \
# include jdk.disabled.namedCurves
But still I'm getting the same error. Someone please help.
I think you're using the wrong property (not only the wrong property but the wrong place as well), you're setting default protocol for HTTPS, while you need to set it for TLS, i.e. add the next line to system.properties file
jdk.tls.client.protocols=TLSv1.2
JMeter restart will be required to apply this property.
If it won't help or you will get different errors - consider adding the next line there as well:
javax.net.debug=all
and then check jmeter.log file and stdout for any suspicious entries
More information:
Configuring JMeter
Apache JMeter Properties Customization Guide
I resolved it by using the latest tibjms.jar in the lib directory in JMeter as the Tibco server was upgraded some hours before I raised this issue.

Let's encrypt SSL with traefick on ECS Fargate

I've been trying to solve this for days, but without any luck:
Situation:
I have a ECS cluster on AWS using Fargate, this cluster contains an instance of Traefick 2.3.4 and other containers. I'm using Traefick as reverse proxy to forward the requests to the other containers.
Using HTTP everything works fine, so I've decided to add also the secure connection to Traefick. I've tried everything that I could find on the Internet but nothing works, when I try to connect to the specified domain with curl it returns:
curl: (35) error:1408F10B:SSL routines:ssl3_get_record:wrong version number
Here there are some test that I've done:
traefick.yml:
log:
level: DEBUG
api:
dashboard: true
entryPoints:
web:
address: :80
http:
redirections:
entryPoint:
to: websecure
scheme: https
websecure:
address: ":443"
providers:
ecs:
clusters:
- tools-cluster
region: eu-west-2
exposedByDefault: false
certificatesResolvers:
letsencrypt:
acme:
caServer: https://acme-staging-v02.api.letsencrypt.org/directory
email: #########################
storage: acme.json
httpchallenge:
entrypoint: web
Labels:
"dockerLabels": {
"traefik.enable": "true",
"traefik.http.services.traefik.loadbalancer.server.port": "8080",
"traefik.http.routers.traefik.rule": "Host(`${host}`)",
"traefik.http.routers.traefik.entrypoints": "websecure",
"traefik.http.routers.traefik.tls.certresolver": "letsencrypt",
"traefik.http.routers.traefik.service": "api#internal"
}
this version returns this error:
rror: 400 :: urn:ietf:params:acme:error:connection :: Fetching https://traefik.baaluu.com/.well-known/acme-challenge/td8IdOvJ1_GkigY-jPYaA4YsgeiS5FUiuUS-avbpsuY: Error getting validation data, url
It tries to retrieve that data but it can't because it is redirected to the https and it can't retrieve because https doesn't work, I've tried also without the auto redirect, and it returns a similar error, it can't retrieve that data.
But following this guide it should work correctly.
So I've decided to move to the dnsChallenge with this configuration:
Traefick.yml
log:
level: DEBUG
api:
dashboard: true
entryPoints:
web:
address: :80
websecure:
address: ":443"
providers:
ecs:
clusters:
- tools-cluster
region: eu-west-2
exposedByDefault: false
certificatesResolvers:
letsencrypt:
acme:
caServer: https://acme-staging-v02.api.letsencrypt.org/directory
email: ######################
storage: acme.json
dnsChallenge:
provider: route53
delayBeforeCheck: 3
and same labels as before:
"dockerLabels": {
"traefik.enable": "true",
"traefik.http.services.traefik.loadbalancer.server.port": "8080",
"traefik.http.routers.traefik.rule": "Host(`${host}`)",
"traefik.http.routers.traefik.entrypoints": "websecure",
"traefik.http.routers.traefik.tls.certresolver": "letsencrypt",
"traefik.http.routers.traefik.service": "api#internal"
}
Still nothing, and I've this inside the logs:AuthURL: https://acme-staging-v02.api.letsencrypt.org/acme/authz-v3/170242259"
That url contains:
{
"type": "urn:ietf:params:acme:error:malformed",
"detail": "Method not allowed",
"status": 405
}
The latest test that I did is to remove the staging ca server:
log:
level: DEBUG
api:
dashboard: true
entryPoints:
web:
address: :80
websecure:
address: :443
providers:
ecs:
clusters:
- tools-cluster
region: eu-west-2
exposedByDefault: false
certificatesResolvers:
letsencrypt:
acme:
email: ###############
storage: acme.json
dnsChallenge:
provider: route53
delayBeforeCheck: 2
The ssl still doesn't work but I don't see any error message inside the logs: this is the last message that I get about a certificate:
Try to challenge certificate for domain [traefik.baaluu.com] found in HostSNI rule" providerName=letsencrypt.acme routerName=traefik#ecs rule="Host(`traefik.baaluu.com`)"
And there is not much more after that:
(I'm sorry for the picture but I don't find a way to extract that logs from ECS)
The other containers are still reachable on the http protocol.
If I try to connect to it using telnet I can reach the service:
telnet traefik.baaluu.com 443
Trying 3.8.30.164...
Connected to traefik-1547500306.eu-west-2.elb.amazonaws.com.
Escape character is '^]'.
Same goes for the 80
Looking better inside the logs I've also find this
retry due to: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/chall-v3/9205340157/1Wh0tQ :: urn:ietf:params:acme:error:badNonce :: JWS has an invalid anti-replay nonce: \"0004cbkFTGjCALFGDYOmhruMl6_F_fRSj33cOMvdpx5Xd2M\", url: "
time="2020-12-10T13:08:21Z" level=debug msg="legolog: [INFO] retry due to: acme: error: 400 :: POST :: https://acme-v02.api.letsencrypt.org/acme/chall-v3/9205340157/1Wh0tQ :: urn:ietf:params:acme:error:badNonce :: JWS has an invalid anti-replay nonce: \"0004cbkFTGjCALFGDYOmhruMl6_F_fRSj33cOMvdpx5Xd2M\", url: "
that contains this url: https://acme-v02.api.letsencrypt.org/acme/chall-v3/9205340157/1Wh0tQ
{
"type": "dns-01",
"status": "valid",
"url": "https://acme-v02.api.letsencrypt.org/acme/chall-v3/9205340157/1Wh0tQ",
"token": "44R4gD4_ZmemiCn5rtkqJyWOcjoj09sEgobUvZLH6yc",
"validationRecord": [
{
"hostname": "traefik.baaluu.com"
}
]
}
So I suppose that the ssl has been generated correctly but I'm not sure.
Any idea or suggestion?
Thanks in advance.
H2K
Edit:
I've removed the ssl from the dashboard and I've put it on another container, now entering inside the dashboard I can see this:
So I suppose that the ssl is working for that domain, but I still can't connect to it.
Edit 2:
with telnet if I connect to that url on the port 443 and I request the page I can see the content:
telnet xxxxxxxxxxxxxxxxx 443
Trying 3.10.148.201...
Connected to traefik-1547500306.eu-west-2.elb.amazonaws.com.
Escape character is '^]'.
GET /index.html HTTP/1.1
Host: xxxxxxxxxxxxxxxxx
And the content of the page appears, so it is not a load balacer problem or routing problem, it seems that I can reach the container using the 443, simply the ssl is not there. It is like to have 2 http port and both are behaving in the same way. The 443 at the moment is like a port 80.
I've have also spent a number of days trying to work it out so i feel your pain.
The error is misleading, the request doesn't even make it past the ALB let alone traefik.
There are two factors to this issue,
The first being that when you specify a port 443 through docker compose as "443:443" you would assume that this creates a HTTPS listener, it actually creates a listener for 443 on the HTTP protocol. In addition the listener also sent the data to the fargate HTTP port and didn't redirect. I'm not sure if this is a bug, or because because i haven't specified that the protocol should be "x-aws-protocol: https" on the target port.
I also found some AWS documentation that said if you use a HTTPS port on a ALB that you need an SSL certificate in place at a ALB level. This kind of makes sense that you can't terminate the connection at a task level if you consider the swarm nature and security implications (better minds are welcome to explain)
With the above in mind i created a certificate in the ACM that covered all the the domains that i needed, changed the listener to the HTTPS protocol and specified the certificate i created. At this point i was able to configure traefik to accept traefik to the frontend.

KAFKA and SSL : java.lang.OutOfMemoryError: Java heap space when using kafka-topics command on KAFKA SSL cluster

this is my first post on Stackoverflow, i hope i didnt choose the wrong section.
Context :
Kafka HEAP size is configured on following file :
/etc/systemd/system/kafka.service
With following parameter :
Environment="KAFKA_HEAP_OPTS=-Xms6g -Xmx6g"
OS is "CentOS Linux release 7.7.1908".
Kafka is "confluent-kafka-2.12-5.3.1-1.noarch", installed from the following repository :
# Confluent REPO
[Confluent.dist]
name=Confluent repository (dist)
baseurl=http://packages.confluent.io/rpm/5.3/7
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/5.3/archive.key
enabled=1
[Confluent]
name=Confluent repository
baseurl=http://packages.confluent.io/rpm/5.3
gpgcheck=1
gpgkey=http://packages.confluent.io/rpm/5.3/archive.key
enabled=1
I activated SSL on a 3-machine KAFKA cluster few days ago, and suddently, the following command stopped working :
kafka-topics --bootstrap-server <the.fqdn.of.server>:9093 --describe --topic <TOPIC-NAME>
Which return me the following error :
[2019-10-03 11:38:52,790] ERROR Uncaught exception in thread 'kafka-admin-client-thread | adminclient-1':(org.apache.kafka.common.utils.KafkaThread)
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.<init>(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:335)
at org.apache.kafka.common.memory.MemoryPool$1.tryAllocate(MemoryPool.java:30)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:112)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:424)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:385)
at org.apache.kafka.common.network.Selector.attemptRead(Selector.java:651)
at org.apache.kafka.common.network.Selector.pollSelectionKeys(Selector.java:572)
at org.apache.kafka.common.network.Selector.poll(Selector.java:483)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:539)
at org.apache.kafka.clients.admin.KafkaAdminClient$AdminClientRunnable.run(KafkaAdminClient.java:1152)
at java.lang.Thread.run(Thread.java:748)
On the server's log, the following line appears when i try to request it via "kafka-topics" :
/var/log/kafka/server.log :
[2019-10-03 11:41:11,913] INFO [SocketServer brokerId=<ID>] Failed authentication with /<ip.of.the.server> (SSL handshake failed) (org.apache.kafka.common.network.Selector)
I was able to use this command properly BEFORE implementing SSL on the cluster. Here is the configuration i'm using.
All functionnality work properly (consumers, producers...) except "kafka-topics" :
# SSL Configuration
ssl.truststore.location=<truststore-path>
ssl.truststore.password=<truststore-password>
ssl.keystore.type=<keystore-type>
ssl.keystore.location=<keystore-path>
ssl.keystore.password=<keystore-password>
# Enable SSL between brokers
security.inter.broker.protocol=SSL
# Listeners
listeners=SSL://<fqdn.of.the.server>:9093
advertised.listeners=SSL://<fqdn.of.the.server>:9093
There is no problem with the certificate (which is signed by internal CA, internal CA which i added to the truststore specified on the configuration). OpenSSL show no errors :
openssl s_client -connect <fqdn.of.the.server>:9093 -tls1
>> Verify return code: 0 (ok)
The following command is working pretty well with SSL, thanks to parameter "-consumer.config client-ssl.properties"
kafka-console-consumer --bootstrap-server <fqdn.of.the.server>:9093 --topic <TOPIC-NAME> -consumer.config client-ssl.properties
"client-ssl.properties" content is :
security.protocol=SSL
ssl.truststore.location=<truststore-path>
ssl.truststore.password=<truststore-password>
Right now, i'm forced to use "--zookeeper", which according to the documentation, is deprecated :
--zookeeper <String: hosts> DEPRECATED, The connection string for
the zookeeper connection in the form
host:port. Multiple hosts can be
given to allow fail-over.
And of course, it's working pretty well :
kafka-topics --zookeeper <fqdn.of.the.server>:2181 --describe --topic <TOPIC-NAME>
Topic:<TOPIC-NAME> PartitionCount:3 ReplicationFactor:2
Configs:
Topic: <TOPIC-NAME> Partition: 0 Leader: <ID-3> Replicas: <ID-3>,<ID-1> Tsr: <ID-1>,<ID-3>
Topic: <TOPIC-NAME> Partition: 1 Leader: <ID-1> Replicas: <ID-1>,<ID-2> Isr: <ID-2>,<ID-1>
Topic: <TOPIC-NAME> Partition: 2 Leader: <ID-2> Replicas: <ID-2>,<ID-3> Isr: <ID-2>,<ID-3>
So, my question is : why am i unable to use "--bootstrap-server" atm ? Because of the "zookeeper" deprecation, i'm worried about not to be able to consult my topics, and their details...
I believe that kafka-topics needs the same option than kafka-console-consumer, aka "-consumer.config"...
Ask if any additionnal precision needed.
Thanks a lot, hope my question is clear and readable.
Blyyyn
I finally found a way to deal with this SSL error. The key is to use the following setting :
--command-config client-ssl.properties
This is working with the most part of KAFKA commands, like kafka-consumer-groups, and of course kafka-topics. See examples below :
kafka-consumer-groups --bootstrap-server <kafka-hostname>:<kafka-port> --group <consumer-group> --topic <topic> --reset-offsets --to-offset <offset> --execute --command-config <ssl-config>
kafka-topics --list --bootstrap-server <kafka-hostname>:<kafka-port> --command-config client-ssl.properties
ssl-config was "client-ssl.properties",see initial post for content.
Beware, by using IP address on , you'll have an error if the machine certificate doesnt have alternative name with that IP address. Try to have correct DNS resolution and use FQDN if possible.
Hope this solution will help, cheers!
Blyyyn
Stop your Brokers and run below ( assuming you have more that 1.5GB RAM on your server)
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
then start your Brokers on all 3 nodes and then try it.
Note that for consumer and producer clients you need to prefix security.protocol accordingly inside your client-ssl.properties.
For Kafka Consumers:
consumer.security.protocol=SASL_SSL
For Kafka Producers:
producer.security.protocol=SASL_SSL

SSL for Phoenix working locally but not on production

I have an Phoenix/Elixir App that works fine with https locally, however when I try to change it to use the production certificates the server does not respond and no error messages are shown.
In my dev.exs this was made with the hostname localhost
In prod.exs here are the keys. These were made with my production URL
I have tried to change the localhost to production url on local by adding host into the https portion in the config
https: [port: 443,
host: "produrl.com"
keyfile: "priv/keys/domain.key",
certfile: "priv/keys/domain.crt"],
This throws an error
sudo MIX_ENV=prod mix phoenix.server
[info] Running LiteChartBe.Endpoint with Cowboy using http://localhost:80
[info] Application lite_chart_be exited: LiteChartBe.start(:normal, []) returned an error: shutdown: failed to start child: LiteChartBe.Endpoint
** (EXIT) shutdown: failed to start child: Phoenix.Endpoint.Server
** (EXIT) shutdown: failed to start child: {:ranch_listener_sup, LiteChartBe.Endpoint.HTTPS}
** (EXIT) shutdown: failed to start child: :ranch_acceptors_sup
** (EXIT) :badarg
{"Kernel pid terminated",application_controller,"{application_start_failure,lite_chart_be,{{shutdown,{failed_to_start_child,'Elixir.LiteChartBe.Endpoint',{shutdown,{failed_to_start_child,'Elixir.Phoenix.Endpoint.Server',{shutdown,{failed_to_start_child,{ranch_listener_sup,'Elixir.LiteChartBe.Endpoint.HTTPS'},{shutdown,{failed_to_start_child,ranch_acceptors_sup,badarg}}}}}}}},{'Elixir.LiteChartBe',start,[normal,[]]}}}"}
Kernel pid terminated (application_controller) ({application_start_failure,lite_chart_be,{{shutdown,{failed_to_start_child,'Elixir.LiteChartBe.Endpoint',{shutdown,{failed_to_start_child,'Elixir.Phoeni
If I simply forward localhost to produrl in my local hosts file, no errors are thrown and nothing connects to the server using https.
The error states that you provided a wrong argument for the configuration of your Endpoint (** (EXIT) :badarg). I suppose that is beacause you are missing a comma behind your host url.
This does probably not solve your problem, but that is supposedly the reason for the error message shown after your change.

External CA configuration with puppet agent

I am trying to configure the puppetserver and agent to use external CA with - Root self-signed CA & Master,Agent having its own ssl certificate
Configurations in puppetserver:
/etc/puppetlabs/puppetserver/bootstrap.cfg
# To enable the CA service, leave the following line uncommented
# puppetlabs.services.ca.certificate-authority-service/certificate-authority-service
# To disable the CA service, comment out the above line and uncomment the line below
puppetlabs.services.ca.certificate-authority-disabled-service/certificate-authority-disabled-service
/etc/puppetlabs/puppetserver/conf.d/webserver.conf
ssl-cert : /usr/cachelogic/var/device-pki/dev_cert.pem
ssl-key : /usr/cachelogic/var/device-pki/dev_key.pem
ssl-ca-cert : /usr/cachelogic/var/device-pki/CAcert.pem
ssl-crl-path : /etc/puppetlabs/puppet/ssl/crl.pem
puppetserver service was started successfully.
Configurations in puppet agent:
/etc/puppetlabs/puppet/puppet.conf
hostcert = /usr/cachelogic/var/device-pki/dev_cert.pem
hostprivkey = /usr/cachelogic/var/device-pki/dev_key.pem
localcacert = /usr/cachelogic/var/device-pki/CAcert.pem
While starting the puppet agent following this the error message that I get.
Debug: Using cached certificate for ca
Debug: Creating new connection for https://cp3.zzz152d1.cdn:8140
Debug: Using cached certificate for ca
Error: Could not run: stack level too deep
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:63
Any pointers on debugging this issue will be helpful. Thanks.