I have a rancher 2.6.67 server and RKE2 downstream cluster. The cluster was created without authorized cluster endpoint. How to add an authorised cluster endpoint to a RKE2 cluster created by Rancher article describes how to add it in an existing cluster, however although the answer looks promising, I still must miss some detail, because it does not work for me.
Here is what I did:
Created /var/lib/rancher/rke2/kube-api-authn-webhook.yaml file with contents:
apiVersion: v1
kind: Config
clusters:
- name: Default
cluster:
insecure-skip-tls-verify: true
server: http://127.0.0.1:6440/v1/authenticate
users:
- name: Default
user:
insecure-skip-tls-verify: true
current-context: webhook
contexts:
- name: webhook
context:
user: Default
cluster: Default
and added
"kube-apiserver-arg": [
"authentication-token-webhook-config-file=/var/lib/rancher/rke2/kube-api-authn-webhook.yaml"
to the /etc/rancher/rke2/config.yaml.d/50-rancher.yaml file.
After restarting rke2-server I found the network configuration tab in Rancher and was able to enable authorized endpoint. Here is where my success ends.
I tried to create a serviceaccount and got the secret to have token authorization, but it failed when connecting directly to the api endpoint on the master.
kube-api-auth pod logs this:
time="2022-10-06T08:42:27Z" level=error msg="found 1 parts of token"
time="2022-10-06T08:42:27Z" level=info msg="Processing v1Authenticate request..."
Also the log is full of messages like this:
E1006 09:04:07.868108 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterAuthToken: failed to list *v3.ClusterAuthToken: the server could not find the requested resource (get clusterauthtokens.meta.k8s.io)
E1006 09:04:40.778350 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterAuthToken: failed to list *v3.ClusterAuthToken: the server could not find the requested resource (get clusterauthtokens.meta.k8s.io)
E1006 09:04:45.171554 1 reflector.go:139] pkg/mod/github.com/rancher/client-go#v1.22.3-rancher.1/tools/cache/reflector.go:168: Failed to watch *v3.ClusterUserAttribute: failed to list *v3.ClusterUserAttribute: the server could not find the requested resource (get clusteruserattributes.meta.k8s.io)
I found that SA tokens will not work this way so I tried to use a rancher user token, but that fails as well:
time="2022-10-06T08:37:34Z" level=info msg=" ...looking up token for kubeconfig-user-qq9nrc86vv"
time="2022-10-06T08:37:34Z" level=error msg="clusterauthtokens.cluster.cattle.io \"cattle-system/kubeconfig-user-qq9nrc86vv\" not found"
Checking the cattle-system namespace, there are no SA and secret entries corresponding to the users created in rancher, however I found SA and secret entries related in cattle-impersonation-system.
I tried creating a new user, but that too, only resulted in new entries in cattle-impersonation-system namespace, so I presume kube-api-auth wrongly assumes the location of the secrets to be cattle-system namespace.
Now the questions:
Can I authenticate with downstream RKE2 cluster using normal SA tokens (not ones created through Rancher server)? If so, how?
What did I do wrong about adding the webhook authentication configuration? How to make it work?
I noticed, that since I made the modifications described above, I cannot download the kubeconfig file from the rancher UI for this cluster. What went wrong there?
Thanks in advance for any advice.
I was trying to add an oauth2.0 authentication provider in our cas (v6.1.x). But I was getting the following error.
2020-11-15 10:03:30,675 INFO [org.apereo.cas.web.flow.DelegatedClientAuthenticationAction] - <Credentials are successfully authenticated using the delegated client [OauthClient]>
2020-11-15 10:03:36,492 ERROR [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <Authentication has failed. Credentials may be incorrect or CAS cannot find authentication handler that supports [ClientCredential(credentials=#OAuth20Credentials# | code: c.lKObb15ip36uiWfOYaTXEfQ | accessToken: com.github.scribejava.core.model.OAuth2AccessToken#5ca28902 |, clientName=OauthClient, typedIdUsed=true, userProfile=null)] of type [ClientCredential]. Examine the configuration to ensure a method of authentication is defined and analyze CAS logs at DEBUG level to trace the authentication event.>
2020-11-15 10:03:36,509 ERROR [org.apereo.cas.authentication.PolicyBasedAuthenticationManager] - <[OauthClient]: [id cannot be blank]>
2020-11-15 10:03:36,513 INFO [org.apereo.inspektr.audit.support.Slf4jLoggingAuditTrailManager] - <Audit trail record BEGIN
=============================================================
WHO: NotYetAuthenticated-e111ad8e-8e6f-4edd-9de7-d2eae5040704
As you can see the credentials are successfully authenticated. After that redirected back with unauthorized access on the browser UI with the above error.
The properties I used for delegated authentication is given below:-
cas.server.name=http://localhost:8080
cas.server.prefix=http://localhost:8080/cas
cas.authn.pac4j.oauth2[0].id=${CLIENT_ID}
cas.authn.pac4j.oauth2[0].secret=${CLIENT_SECRET}
cas.authn.pac4j.typedIdUsed=true
cas.authn.pac4j.principalAttributeId=email
cas.authn.pac4j.name=OauthClient
cas.authn.pac4j.order=0
cas.authn.pac4j.lazyInit=true
cas.authn.pac4j.oauth2[0].autoRedirect=false
cas.authn.pac4j.oauth2[0].principalAttributeId=email
cas.authn.pac4j.oauth2[0].enabled=true
cas.authn.pac4j.oauth2[0].authUrl=${AUTH_URL}
cas.authn.pac4j.oauth2[0].tokenUrl=${TOKEN_URL}
cas.authn.pac4j.oauth2[0].profileUrl=${PROFILE_URL}
cas.authn.pac4j.oauth2[0].profileVerb=GET
cas.authn.pac4j.oauth2[0].scope=profile,email,roles
cas.authn.pac4j.oauth2[0].clientName=OauthClient
Thanks in advance.
I have a Vault server backed by a Consul cluster and integrated with my LDAP server, it works fine with my LDAP server and every thing goes well with it, but the only thing is I cannot renew the tokens generated by these logins.
To Reproduce
Steps to reproduce the behaviour:
Run vault login -method=ldap username=myusername -renewable=true and get the token as following:
Password (will be hidden):
Success! You are now authenticated. The token information displayed below
is already stored in the token helper. You do NOT need to run "vault login"
again. Future Vault requests will automatically use this token.
Key Value
--- -----
token s.wCQedkMmX61EJszE64HqPzhC
token_accessor qcxkggK00WxgwmxOC9Ht9vpc
token_duration 24h
token_renewable true
token_policies ["default"]
identity_policies []
policies ["default"]
token_meta_username myusername
Login as root user and Run vault token lookup s.wCQedkMmX61EJszE64HqPzhC to check token status and ttl:
Key Value
--- -----
accessor qcxkggK00WxgwmxOC9Ht9vpc
creation_time 1576051650
creation_ttl 24h
display_name ldap-myusername
entity_id 1fc1f68d-face-f9f1-468f-36b94e10fb3b
expire_time 2019-12-12T08:07:30.56805754Z
explicit_max_ttl 0s
id s.wCQedkMmX61EJszE64HqPzhC
issue_time 2019-12-11T08:07:30.568070919Z
meta map[username:myusername]
num_uses 0
orphan true
path auth/ldap/login/myusername
policies [default]
**renewable true**
ttl 23h55m5s
type service
As it is obvious the renewable property of the token is true and its type is service, so it can be renewed.
Run vault token renew s.wCQedkMmX61EJszE64HqPzhC to renew the token given above.
When I look up the token again nothing happened to its ttl. Run vault token lookup s.wCQedkMmX61EJszE64HqPzhC:
Key Value
--- -----
accessor qcxkggK00WxgwmxOC9Ht9vpc
creation_time 1576051650
creation_ttl 24h
display_name ldap-myusername
entity_id 1fc1f68d-face-f9f1-468f-36b94e10fb3b
expire_time 2019-12-12T08:07:30.56805754Z
explicit_max_ttl 0s
id s.wCQedkMmX61EJszE64HqPzhC
issue_time 2019-12-11T08:07:30.568070919Z
meta map[username:myusername]
num_uses 0
orphan true
path auth/ldap/login/myusername
policies [default]
renewable true
ttl 23h53m24s
type service
Note: I tried the steps above using API calls and self-renew but the result was same as above.
Expected behavior
My expected behaviour was after running vault token renew s.wCQedkMmX61EJszE64HqPzhC for a LDAP token as root the ttl of the token gets back to creation_ttl vaule.
Environment:
Vault Server Version:
root#ubuntu:~# vault status
Key Value
--- -----
Seal Type shamir
Initialized true
Sealed false
Total Shares 5
Threshold 3
Version 1.3.0
Cluster Name vault-cluster-11d62d58
Cluster ID a9704841-7f1c-1986-a880-a2c252f23ed2
HA Enabled true
HA Cluster https://10.1.10.1:8201
HA Mode active
Vault CLI Version:
root#ubuntu:~# vault version
Vault v1.3.0
Server Operating System/Architecture:
My OS is Ubuntu 18.04 with this info:
root#ubuntu:~# uname -a
Linux ubuntu 4.15.0-45-generic #48-Ubuntu SMP Tue Jan 29 16:28:13 UTC 2019 x86_64 x86_64 x86_64 GNU/Linux
Vault server configuration file(s):
listener "tcp" {
address = "0.0.0.0:8200"
cluster_address = "10.1.10.1:8201"
tls_disable = "true"
}
storage "consul" {
address = "127.0.0.1:8500"
path = "vault/"
}
ui = true
api_addr = "http://10.1.10.1:8200"
cluster_addr = "https://10.1.10.1:8201"
UPDATE:
You can use this sample free LDAP server config. to reproduce the situation:
#Test LDAP server
vault write auth/ldap/config \
url="ldap://ldap.forumsys.com:389" \
userdn="uid=tesla,dc=example,dc=com" \
userattr="uid" \
groupattr="cn" \
groupdn="dc=example,dc=com" \
binddn="uid=tesla,dc=example,dc=com" \
bindpass='password' \
starttls=false
login using: vault login -method=ldap username=tesla and password as password and then try to renew the generated token.
The LDAP auth backend's max TTL may be set at 24h. This means that tokens generated cannot live past 24h from its creation.
See the TTL by running
vault auth list --detailed
If the value is system the default value is 32 days or the value specified in the Vault configuration file.
Max TTL can be tuned by:
vault mount-tune -max-lease-ttl=<NEW TTL> auth/ldap
Additional info here
The problem was a bug on version 1.3.0 of Vault, I have created an issue for the bug, which leads to a PR for the next version, and the problem got fixed on 1.3.2.
I'm trying to set up Microsoft AD like user repository for IBM MQ v9 Queue Manager , but without success. I read the document https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_9.0.0/com.ibm.mq.ref.adm.doc/q085490_.htm, but it's very unclear with all those diagrams, dashes and arrows. My final goal is to have ability to grant or rewoke authorizations based od AD groups. Can someone give me complete commands example how to configure queue manager to use AD for user repository?
IBM MQ is v9.0.0.0 and runs on CentOS v7. Active Directory is on Windows Server 2019 machine.
I tried to set AUTHINFO with MQSC commands. All commands are executed without problems. After that I refreshed security and tried to grant authorizations with setmqaut command, but unsuccessful.
I tried with this below MQSC commands:
DEFINE AUTHINFO(MY.AD.CONFIGURATION) AUTHTYPE(IDPWLDAP) AUTHORMD(SEARCHGRP) FINDGRP(member) CONNAME('192.168.100.100') BASEDNG('OU=Groups,OU=MyCompany,DC=mycompany,DC=us') SHORTUSR('sAMAccountName') LDAPUSER('mybinduser') LDAPPWD('mypassword')
ALTER QMGR CONNAUTH(MY.AD.CONFIGURATION)
REFRESH SECURITY TYPE(CONNAUTH)
setmqaut -m MY.QUEUE.MANAGER -t qmgr -g myadgroup +all
After I execute command:
setmqaut -m MY.QUEUE.MANAGER -t qmgr -g myadgroup +all
This error is displyed i console: AMQ7026: A principal or group name was invalid.
And these below lines are recorded in queue manager log:
AMQ5531: Error locating user or group in LDAP
EXPLANATION:
The LDAP authentication and authorization service has failed in the ldap_search
call while trying to find user or group 'myadgroup '. Returned count is 0.
Additional context is 'rc = 87 (Bad search filter)
[(&(objectClass=groupOfNames)(=myadgroup ))]'.
ACTION:
Specify the correct name, or fix the directory configuration. There may be
additional information in the LDAP server error logs.
----- amqzfula.c : 2489 -------------------------------------------------------
On Active Directory side these lines are recorded in log:
An account failed to log on.
Subject:
Security ID: SYSTEM
Account Name: MYADSERVER$
Account Domain: MYDOMAINNAME
Logon ID: 0x3E7
Logon Type: 3
Account For Which Logon Failed:
Security ID: NULL SID
Account Name: mybinduser
Account Domain: MYDOMAINNAME
Failure Information:
Failure Reason: Unknown user name or bad password.
Status: 0xC000006D
Sub Status: 0xC000006A
Process Information:
Caller Process ID: 0x280
Caller Process Name: C:\Windows\System32\lsass.exe
Network Information:
Workstation Name: MYADSERVER
Source Network Address: 192.168.100.101
Source Port: 55592
Detailed Authentication Information:
Logon Process: Advapi
Authentication Package: MICROSOFT_AUTHENTICATION_PACKAGE_V1_0
Transited Services: -
Package Name (NTLM only): -
Key Length: 0
Here beleow is output of the command DIS AUTHINFO(MY.AD.CONFIGURATION) ALL
AMQ8566: Display authentication information details.
AUTHINFO(MY.AD.CONFIGURATION) AUTHTYPE(IDPWLDAP)
ADOPTCTX(NO) DESCR( )
CONNAME(192.168.100.100) CHCKCLNT(REQUIRED)
CHCKLOCL(OPTIONAL) CLASSGRP( )
CLASSUSR( ) FAILDLAY(1)
FINDGRP(MEMBER) BASEDNG(OU=Groups,OU=MyCompany,DC=mycompany,DC=us)
BASEDNU( )
LDAPUSER(CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us)
LDAPPWD( ) SHORTUSR(sAMAccountName)
GRPFIELD( ) USRFIELD( )
AUTHORMD(SEARCHGRP) NESTGRP(NO)
SECCOMM(NO) ALTDATE(2019-07-25)
ALTTIME(08.14.20)
Here below is output from LdapAuthentication.jar tool:
java -jar LdapAuthentication.jar ldap://192.168.100.100:389 CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us mybinduserpassword OU=MyCompany,DC=mycompany,DC=us sAMAccountName adminusername adminpassword
#WMBL3: successful bind
#WMBL3: successfull search Starting Authentication Found the user, DN is CN=adminusername,OU=MyCompany,OU=Users,OU=MyCompany,DC=mycompany,DC=us
#WMBL3 : check if the password is correct
#WMBL3: successful authentication
#WMBL3 : Commands for WebUI ldap authentication :
1. mqsisetdbparms <INodeName> -n ldap::LDAP -u "CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us" -p mybinduserpassword
Or
mqsisetdbparms <INodeName> -n ldap::192.168.100.100 -u "CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us" -p mybinduserpassword
2. mqsichangeproperties <INodeName> -b webadmin -o server -n ldapAuthenticationUri -v \"ldap://192.168.100.100:389/OU=MyCompany,DC=mycompany,DC=us?sAMAccountName\"
3. mqsiwebuseradmin <INodeName> -c -u adminusername -x -r <sysrole for eg: local userid >
Here below is qmanager log after I applied changes in my AUTHINFO what you suggested Jul 25.
AMQ5531: Error locating user or group in LDAP
EXPLANATION:
The LDAP authentication and authorization service has failed in the ldap_search
call while trying to find user or group 'wasadmin'. Returned count is 0.
Additional context is 'rc = 1 (Operations error)
[(&(objectClass=GROUP)(SAMACCOUNTNAME=wasadmin))]'.
ACTION: Specify the correct name, or fix the directory configuration. There may be
additional information in the LDAP server error logs.
This is myadgroup full DN:
CN=myadgroup,OU=System,OU=Groups,OU=MyCompany,DC=mycompany,DC=us
This is output of the setmqaut command with full group DN:
setmqaut -m MY.QUEUE.MANAGER -t qmgr -g 'CN=myadgroup,OU=System,OU=Groups,OU=MyCompany,DC=mycompany,DC=us' +all
AMQ7047: An unexpected error was encountered by a command. Reason code is 2063.
And this is qmanager log after that command was executed:
AMQ5531: Error locating user or group in LDAP
EXPLANATION: The LDAP authentication and authorization service has failed in the ldap_search call while trying to find user or group 'CN=myadgroup,OU=System,OU=Groups,OU=MyCompany,DC=mycompany,DC=us'.
Returned count is 0.
Additional context is 'rc = 1 (Operations error) [(objectClass=groupOfNames)]'.
ACTION:
Specify the correct name, or fix the directory configuration. There may be
additional information in the LDAP server error logs.
If I try with CLASSGRP(GROUP) output of the setmqaut is:
AMQ7047: An unexpected error was encountered by a command. Reason code is 2063.
And qmqnager log is:
AMQ5531: Error locating user or group in LDAP
EXPLANATION: The LDAP authentication and authorization service has failed in the
ldap_search call while trying to find user or group
'CN=myadgroup,OU=System,OU=Groups,OU=MyCompany,DC=mycompany,DC=us'.
Returned count is 0.
Additional context is 'rc = 1 (Operations error) [(objectClass=GROUP)]'.
ACTION:
Specify the correct name, or fix the directory configuration. There may be
additional information in the LDAP server error logs.
Below is my last configured authinfo object:
AMQ8566: Display authentication information details.
AUTHINFO(MY.AD.CONFIGURATION) AUTHTYPE(IDPWLDAP)
ADOPTCTX(YES) DESCR( )
CONNAME(192.168.100.100) CHCKCLNT(OPTIONAL)
CHCKLOCL(OPTIONAL) CLASSGRP(group)
CLASSUSR(USER) FAILDLAY(1)
FINDGRP(member)
BASEDNG(OU=Groups,OU=MyCompany,DC=mycompany,DC=us)
BASEDNU(OU=Users,OU=MyCompany,DC=mycompany,DC=us)
LDAPUSER(CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us)
LDAPPWD( ) SHORTUSR(sAMAccountName)
GRPFIELD(sAMAccountName) USRFIELD(sAMAccountName)
AUTHORMD(SEARCHGRP) NESTGRP(NO)
SECCOMM(NO) ALTDATE(2019-08-07)
ALTTIME(08.44.40)
Based on the your output I noted that you did not set LDAPPWD which is used by MQ to authenticate the LDAPUSER that you specified.
This is supported by the windows error you provided:
Account For Which Logon Failed:
Security ID: NULL SID
Account Name: mybinduser
Account Domain: MYDOMAINNAME
Failure Information:
Failure Reason: Unknown user name or bad password.
In the output of LdapAuthentication.jar it appears that you have the correct password available:
CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us mybinduserpassword
You can either specify the LDAPPWD or you can blank out your LDAPUSER and see if your AD allows anonymous bind (this is rare).
I noted that you have some other fields left blank that probably need to be filled in. I also suggest you always use ADOPTCTX(YES).
Below is my suggested updates to your AUTHINFO object:
ALTER AUTHINFO(MY.AD.CONFIGURATION) +
AUTHTYPE(IDPWLDAP) +
AUTHORMD(SEARCHGRP) +
FINDGRP('member') +
ADOPTCTX(YES) +
CONNAME(192.168.100.100) +
CHCKCLNT(REQUIRED) +
CHCKLOCL(OPTIONAL) +
CLASSGRP(GROUP) +
CLASSUSR(USER) +
FAILDLAY(1) +
BASEDNG('OU=MyCompany,DC=mycompany,DC=us') +
BASEDNU('OU=MyCompany,DC=mycompany,DC=us') +
LDAPUSER('CN=mybinduser,OU=System,OU=Users,OU=MyCompany,DC=mycompany,DC=us') +
LDAPPWD(mybinduserpassword) +
SHORTUSR(sAMAccountName) +
GRPFIELD(sAMAccountName) +
USRFIELD(sAMAccountName) +
NESTGRP(NO) +
SECCOMM(NO)
*Note I have not tested this against AD, but I have setup IIB to authenticate the WebUI/REST calls against AD and also took inspiration from two presentations/write ups from Mark Taylor from IBM:
MQ Integration with Directory Services - Presented at MQTC v2.0.1.6
MQdev Blog: IBM MQ - Using Active Directory for authorisation in Unix queue managers
We have put these entries in hive-site.xml:
hive.server2.authentication : KERBEROS
hive.server2.authentication.kerberos.keytab : /tmp/hive.keytab
hive.server2.authentication.kerberos.principal : hive/FQDN of the hive VM#xxxxxxxx.COM
Using kinit command on the hive VM, we have verified that Kerberos principal and the keytab file are valid:
kinit -t FILE:/tmp/hive.keytab -k hive/FQDN of the hive VM#xxxxxxxx.COM
Then if we do,
klist
it shows the same in Ticket Cache as the default Principal.
But, when we try to start the HiveServer2 using :
sudo service hive-server2 start
it throws the exception :
Starting HiveServer2
javax.security.auth.login.LoginException: Kerberos principal should have 3 parts: hive
at org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:127)
at org.apache.hive.service.cli.thrift.ThriftCLIService.run(ThriftCLIService.java:505)
at java.lang.Thread.run(Thread.java:679)
When we try to start the service (using ./hiveserver2) with any other logged in user, say User123, it throws the same exception with :
Starting HiveServer2
javax.security.auth.login.LoginException: Kerberos principal should have 3 parts: User123
at org.apache.hive.service.auth.HiveAuthFactory.getAuthTransFactory(HiveAuthFactory.java:127)
at org.apache.hive.service.cli.thrift.ThriftCLIService.run(ThriftCLIService.java:505)
at java.lang.Thread.run(Thread.java:679)
Shouldn’t Kerberos Principal be picked up from the hive-site.xml and not the login user? Are we missing out something.
--
I have created a principal hive/FQDN of the hive VM#xxxxxxxx.COM in advance and created a keytab file for it.
We are on CDH 4.7 (not installed using CM), OEL6 and Kerberos5
Kerberos secuirty should be configured for HDFS and MR too, and not just HIVE.