ELK - X-Pack Custom realm - authentication

Question

I've developed a custom realm for my ELK cluster.
This module works well on a on node elasticsearch but when I install it on my production cluster, nothing works.
Elasticsearch starting logs :
- nothing special, everything seems to work and xpack module is loaded (generates a log in stdout)
Elasticsearch cluster diagnostic (custom seems to be disabled and not available) :
{
"security" : {
"available" : true,
"enabled" : true,
"realms" : {
"file" : {
"available" : true,
"enabled" : false
},
"ldap" : {
"available" : true,
"enabled" : false
},
"native" : {
"name" : [
"realm2"
],
"available" : true,
"size" : [
2
],
"enabled" : true,
"order" : [
1
]
},
"custom" : {
"available" : false,
"enabled" : false
},
...
}
Elasticsearch configuration :
cluster.name: "production-cluster-1"
network.host: 0.0.0.0
bootstrap.memory_lock: true
xpack.security.enabled: true
xpack.security.audit.enabled: true
xpack.monitoring.enabled: true
xpack.graph.enabled: false
xpack.watcher.enabled: false
xpack.ml.enabled: false
discovery.zen.ping.unicast.hosts: "-------------------"
network.publish_host: "----"
discovery.zen.minimum_master_nodes: 3
xpack.security:
authc:
realms:
realm1:
type: custom
order: 0
realm2:
type: native
order: 1
The native authentication works fine.
How can I troubleshoot this correctly ? :)
Thanks

Answer 1

Let's start by turning up the logging:
curl -u<user> -XPUT '<host>:<http-port>/_cluster/settings?pretty' -H 'Content-Type: application/json' -d'
{
"transient": {
"logger.org.elasticsearch.xpack.security.authc": "DEBUG"
}
}
'
That will give you any authentication logs on debug. I think the right package for the native realm should be logger.org.elasticsearch.xpack.security.authc.esnative in case you want to limit it down just to that.