Unable to run kafka with self-signed certificates - ssl

Question

I am setting up kafka that uses SASL_PLAIN and SSL auth. I set one up in a public vpc so that I could use certbot to generate certs, but for this one I am setting it in a private network that cannot be accessed by certbot (and I cannot allow it to be accessed either).
So I want to use self-signed certs to do this. I've tried this:
openssl req -new -newkey rsa:4096 \
-days 3650 \
-x509 \
-subj "/CN=$(hostname)" \
-keyout key.pem \
-out cert.pem \
-passout "pass:${PASSWORD}"
openssl pkcs12 -export -out certout -name kafka \
-inkey "key.pem" \
-in "cert.pem" \
-password "pass:${PASSWORD}" \
-passin "pass:${PASSWORD}"
keytool -importkeystore -noprompt \
-srckeystore certout \
-srcstoretype pkcs12 \
-destkeystore /etc/ssl/cert.jks \
-deststoretype pkcs12 \
-srcstorepass "${PASSWORD}" \
-deststorepass "${PASSWORD}"
And in /etc/kafka/server.properties I have:
ssl.keystore.location=/etc/ssl/cert.jks
ssl.truststore.location=/etc/ssl/certs/java/cacerts
Where cacerts comes from is the apt package ca-certificates-java (and I ran update-ca-certificates -f too)
And if I try keytool -import -alias kafka -file certout -cacerts it gives the error
keytool error: java.io.IOException: Keystore was tampered with, or password was incorrect
In the logs for kafka, I see every broker spewing this:
INFO [Controller id=2, targetBrokerId=3] Failed authentication with 3.kafka.my.dns/10.1.1.1 (SSL handshake failed) (org.apache.kafka.common.network.Selector)
(I have changed the URL and IP address in that log)
How can I run kafka with self-signed certs?