I am trying to set up WLS dynamic cluster on two machines.
Two nodes are up and running on one of machine where admin server is hosted as well, but when I try to start the node which is on different machine that I have added afterwards I see below exception.
<Jun 7, 2016 2:13:07 AM PDT> <Critical> <Security> <BEA-090518> <Could not decrypt the username attribute value of {AES}Q64tW2ys+PviYQPkPGPc8/c79/RwfgrsoekwDFpgZKI= from the file /usr/home/devtools/Middleware/user_projects/domains/v12C_d/servers/Cluster-0-abc-4/data/nodemanager/boot.properties. If an encrypted attribute was copied from boot.properties from another domain into /usr/home/devtools/Middleware/user_projects/domains/v12C_d/servers/Cluster-0-abc-4/data/nodemanager/boot.properties, change the encrypted attribute to its clear text value, and then restart the server. The attribute will be encrypted again. Otherwise, change all encrypted attributes to their clear text values, then restart the server. All encryptable attributes will be encrypted again. The decryption failed with the exception weblogic.security.internal.encryption.EncryptionServiceException: com.rsa.jsafe.JSAFE_PaddingException: Invalid padding..>
<Jun 7, 2016 2:13:07 AM PDT> <Critical> <Security> <BEA-090518> <Could not decrypt the password attribute value of {AES}qusooByFxC/eTogSMU2YEjfnWRpY69f6MfTeqhqfIFk= from the file /usr/home/devtools/Middleware/user_projects/domains/v12C_d/servers/Cluster-0-abc-4/data/nodemanager/boot.properties. If an encrypted attribute was copied from boot.properties from another domain into /usr/home/devtools/Middleware/user_projects/domains/v12C_d/servers/Cluster-0-abc-4/data/nodemanager/boot.properties, change the encrypted attribute to its clear text value, and then restart the server. The attribute will be encrypted again. Otherwise, change all encrypted attributes to their clear text values, then restart the server. All encryptable attributes will be encrypted again. The decryption failed with the exception weblogic.security.internal.encryption.EncryptionServiceException: com.rsa.jsafe.JSAFE_PaddingException: Invalid padding..>
Enter username to boot WebLogic server:<Jun 7, 2016 2:13:09 AM PDT> <Info> <Management> <BEA-141307> <Unable to connect to the Administration Server. Waiting 5 second(s) to retry (attempt number 1 of 3).>
<Jun 7, 2016 2:13:14 AM PDT> <Info> <Management> <BEA-141307> <Unable to connect to the Administration Server. Waiting 5 second(s) to retry (attempt number 2 of 3).>
On doing a search on internet I saw solutions such as giving the clear userid/password of weblogic admin in boot.properties file and upon restart the userid/password would get encrypted and issue should be fixed.
Well I have tried that and that didn't fix my issue.
Please note that I am using dynamic cluster which suggests that configurations are usually copied over nodes based on server templates.
Will really appreciate any input on this.
Suggestion:
1) Scale down your cluster to a single instance where the boot.properties files is known to work.
2) Change it to clear text.
3) Bounce WebLogic so it get's the file encrypted once again.
4) Make sure it works.
5) Scale Up your cluster again and see if the error persist.
Related
I am stuck at using SSL in IBM Websphere MQ (9.2).
I am building a client library for MQ and to get more familiar with MQ on the server side I have installed IBM MQ Developer edition and ran the supplied scripts to create a 'default' MQ server instance.
Created an client connection for the DEV.APP.SVRCONN server connection
Created a personal certificate by using the IBM Key management tool and named it ibmwebspheremq
Enabled SSL on the Queue Manager (QM1) and labelled it ibmwebspheremq
Updated the SSL configuration for the DEV.APP.SVRCONN channel and set the cipherspec property to TLS 1.2, 256-bit Secure Hash Algorithm, 128-bit AES encryption (TLS_RSA_WITH_AES_128_CBC_SHA256) and made SSL required.
Tested my settings with:
amqssslc -l ibmwebspheremq -k C:\ProgramData\IBM\MQ\qmgrs\QM1\ssl\key -c DEV.APP.SVRCONN -x 127.0.0.1 -s TLS_RSA_WITH_AES_128_CBC_SHA256 -m QM1
And that gave me:
Sample AMQSSSLC start
Connecting to queue manager QM1
Using the server connection channel DEV.APP.SVRCONN
on connection name 127.0.0.1.
Using SSL CipherSpec TLS_RSA_WITH_AES_128_CBC_SHA256
Using SSL key repository stem C:\ProgramData\IBM\MQ\qmgrs\QM1\ssl\key
Certificate Label: ibmwebspheremq
No OCSP configuration specified.
MQCONNX ended with reason code 2035
Error details (from log):
The active values of the channel were 'MCAUSER(app) CLNTUSER(Wilko)
SSLPEER(SERIALNUMBER=61:9B:A4:3E,CN=DESKTOP-ROH98N2,C=NL)
SSLCERTI(CN=DESKTOP-ROH98N2,C=NL) ADDRESS(DESKTOP-ROH98N2)'. The
MATCH(RUNCHECK) mode of the DISPLAY CHLAUTH MQSC command can be used to
identify the relevant CHLAUTH record.
ACTION:
Ensure that the application provides a valid user ID and password, or change
the queue manager connection authority (CONNAUTH) configuration to OPTIONAL to
allow client applications to connect which have not supplied a user ID and
password.
----- cmqxrmsa.c : 2086 -------------------------------------------------------
22/11/2021 15:51:37 - Process(15880.45) User(MUSR_MQADMIN) Program(amqrmppa.exe)
Host(DESKTOP-ROH98N2) Installation(Installation1)
VRMF(9.2.3.0) QMgr(QM1)
Time(2021-11-22T14:51:37.594Z)
CommentInsert1(DEV.APP.SVRCONN)
CommentInsert2(15880(1112))
CommentInsert3(127.0.0.1)
AMQ9999E: Channel 'DEV.APP.SVRCONN' to host '127.0.0.1' ended abnormally.
EXPLANATION:
The channel program running under process ID 15880(1112) for channel
'DEV.APP.SVRCONN' ended abnormally. The host name is '127.0.0.1'; in some cases
the host name cannot be determined and so is shown as '????'.
ACTION:
Look at previous error messages for the channel program in the error logs to
determine the cause of the failure. Note that this message can be excluded
completely or suppressed by tuning the "ExcludeMessage" or "SuppressMessage"
attributes under the "QMErrorLog" stanza in qm.ini. Further information can be
found in the System Administration Guide.
----- amqrmrsa.c : 630 --------------------------------------------------------
I am kind of stuck, I also saw in the log that there is PEER related info dumped, but I am not sing the SSLPEER settings (I just want to let everyone connect with the same certificate).
EDIT 2:
Output from RUNMQSC QM1 and command DISPLAY QMGR CONNAUTH:
1 : DISPLAY QMGR CONNAUTH
AMQ8408I: Display Queue Manager details.
QMNAME(QM1) CONNAUTH(DEV.AUTHINFO)
Output from RUNMQSC QM1 and command DISPLAY AUTHINFO(name-from-previous-command):
3 : DISPLAY AUTHINFO(DEV.AUTHINFO)
AMQ8566I: Display authentication information details.
AUTHINFO(DEV.AUTHINFO) AUTHTYPE(IDPWOS)
ADOPTCTX(YES) DESCR( )
CHCKCLNT(REQDADM) CHCKLOCL(OPTIONAL)
FAILDLAY(1) AUTHENMD(OS)
ALTDATE(2021-11-18) ALTTIME(15.09.20)
Output from DISPLAY CHLAUTH(*):
4 : DISPLAY CHLAUTH(*)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.ADMIN.SVRCONN) TYPE(USERMAP)
CLNTUSER(admin) USERSRC(CHANNEL)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.ADMIN.SVRCONN) TYPE(BLOCKUSER)
USERLIST(nobody)
AMQ8878I: Display channel authentication record details.
CHLAUTH(DEV.APP.SVRCONN) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(CHANNEL)
CHCKCLNT(REQUIRED)
AMQ8878I: Display channel authentication record details.
CHLAUTH(SYSTEM.ADMIN.SVRCONN) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(CHANNEL)
AMQ8878I: Display channel authentication record details.
CHLAUTH(SYSTEM.*) TYPE(ADDRESSMAP)
ADDRESS(*) USERSRC(NOACCESS)
I was expecting not having to provide username and password when using certificates. What am I missing here?
Your queue manager is configured to mandate passwords for any client connections that are trying to run with a resolved MCAUSER that is privileged. That is what CHCKCLNT(REQDADM) on your AUTHINFO(DEV.AUTHINFO) does.
In addition, your CHLAUTH rule for the DEV.APP.SVRCONN channel has upgraded this further to mandate passwords for ALL connections using that channel.
If your intent is to have channels that supply a certificate not be subject to this mandate, then you should add a further, more specific, CHLAUTH rule, something along these lines:-
SET CHLAUTH(DEV.APP.SVRCONN) TYPE(SSLPEERMAP) +
SSLPEER('SERIALNUMBER=61:9B:A4:3E,CN=DESKTOP-ROH98N2,C=NL') +
SSLCERTI('CN=DESKTOP-ROH98N2,C=NL') CHCKCLNT(ASQMGR) USERSRC(CHANNEL)
Bear in mind that if this connection is asserting a privileged user id, it will still be required to supply a password from the system-wide setting of CHCKCLNT(REQDADM).
Remember, if you are ever unsure which CHLAUTH rule you are matching against, all those details you saw in the error message can be used to form a DISPLAY CHLAUTH command to discover exactly which rule you have matched. Read more about that in I’m being blocked by CHLAUTH – how can I work out why?
I have a WebLogic domain whit an Admin server and one Managed Server. I installed the certificate for ssl connection beetwen Admin and Managed, but when I restart the Managed I have this error:
<May 2, 2016 6:39:24 PM CEST> <Info> <Management> <BEA-141307> <Unable to connect to the Administration Server. Waiting 5 second(s) to retry (attempt number 3 of 3).>
<May 2, 2016 6:39:29 PM CEST> <Info> <Management> <BEA-141298> <Could not register with the Administration Server: java.rmi.RemoteException: [Deployer:149150]An IOException occurred while reading the input.; nested exception is:
javax.net.ssl.SSLHandshakeException: General SSLEngine problem>
I changed this option:
admin console Servers -> server name -> Configuration SSL tab -> Advanced -> Change Hostname Verification dropdown to None
Version 12.2.1
Do you have any solutions?
Thanks
Fabrizio
There's probably some problem with the SSL certificate. I would recommend to double check that and also rerun with -Dssl.debug added to the Weblogic command line to get more information. Then check both the admin and managed server log files, as it might have enough information to answer. If not, please and add more information here.
I am getting the following WARN message while I start my host which is one of the Host Controller (HC) that is attached to the Domain Controller(DC).
[Server:server-two] 14:06:13,822 WARN [org.jboss.as.txn] (ServerService Thread Pool -- 33) JBAS010153: Node identifier property is set to the default value. Please make sure it is unique.
And my host-slave.xml has the following config...
<server-identities>
<!-- Replace this with either a base64 password of your own, or use a vault with a vault expression -->
<secret value="c2xhdmVfdXNlcl9wYXNzd29yZA=="/>
</server-identities>
I hope this config is the reason...... maybe I didn't understand..... but I couldn't find node identifier property rather this is the default secret value which I hope could be the cause of this WARN message.
However, I didn't mention HC to lookup host-slave.xml..... the command which I ran to start my HC is.....
[host-~-\-\-\bin]$./domain.sh -Djboss.domain.master.address=nnn.nn.nn.88 -b nnn.nn.nn.89 -bmanagement nnn.nn.nn.89 &
nnn.nn.nn.88 is my DC
Else please advise what's cause of the WARN message.
And please let me know the implication of this WARN message and advise us on the required config to overcome and sort out any consecutive consequences that would've been bound for this WARN.
I'm new to wildfly, and noticed this warning when I started it standalone from eclipse (I'm doing the following tutorial: https://wwu-pi.github.io/tutorials/lectures/eai/020_tutorial_jboss_project.html)
The fix was to add a node-identifier to the core-environment in the subsystem:
<subsystem xmlns="urn:jboss:domain:transactions:2.0">
<core-environment node-identifier="meindertwillemhoving">
<process-id>
<uuid/>
</process-id>
</core-environment>
<recovery-environment socket-binding="txn-recovery-environment" status-socket-binding="txn-status-manager"/>
</subsystem>
This is in file [wildfly]\standalone\configuration\standalone.xml.
This is the same answer as https://developer.jboss.org/message/880136#880136
According to WFLY-10541 if you are using WildFly v14.0.0 or newer you can pass the following to the startup script to set the transaction node identifier:
-Djboss.tx.node.id=<some-unique-id>
Setting the node identifier to an unique value is only required for proper handling of XA Transactions.
You can set it as follows in your XML configuration:
<subsystem xmlns="urn:jboss:domain:transactions:6.0">
<core-environment node-identifier="${jboss.tx.node.id}">
It needs to be a unique value up to 23 bytes long.
More about this here: http://www.mastertheboss.com/jboss-server/jboss-configuration/configuring-transactions-jta-using-jboss-as7-wildfly
Building on #kaptan's answer I added the following to the bottom of
bin/standalone.conf:
JAVA_OPTS="$JAVA_OPTS -Djboss.tx.node.id=`hostname -f`
This way I don't have to remember to add the "-Djboss.tx.node.id=" when running up wildfly by hand.
For this <server-identities> is not the issue. In fact, it shouldn't be touched at all.
When JBoss is started in domain mode by domain.sh, by default there will be three servers server-one server-two server-three. When you are running one more HC attached to the DC.... the defaulted server which is in auto-start mode will get clash when we start HC attaching to DC,- by the following command.
[host-~-\-\-\bin]$./domain.sh -Djboss.domain.master.address=nnn.nn.nn.88 -b nnn.nn.nn.89 -bmanagement nnn.nn.nn.89 &
Or by having the host configuration at HC (default host.xml... until unless we choose a different one....).
<domain-controller>
<remote host="${jboss.domain.master.address:nnn.nn.nn.88}" port="${jboss.domain.master.port:9999}" security-realm="ManagementRealm"/>
<domain-controller>
In order to solve this, we need to turn auto-start to false..... And we need to create a new server-group...... To that group we need to add dc-created-server and hc-created-server..... we can choose the appropriate same profile either full-ha or full for both created servers across DC and HC.
SO when we start the group by configuring the required HEAP size including permgen space... You could start both DC and HC.... and in DC you could see both of your-created-servers are started in the created server-group.
DC- Domain Controller
HC- Host Controller
To deploy you need to upload .ear or web-archive in the Application Console. You cannot place it in the deployments folder as how you do in standalone mode with .dodeploy file.
If you upload the same .ear next version do the Replace option instead of the Remove & Add option in the upload process.
We are trying to use one of the existing weblogic 12c domain and It's DataSource passwords been expired.
Since AdminServer is not responding correctly I tried to re-start AdminServer as well, Now I have changed the DB passwords and wanted to set the new passwords starting the AdminServer, but I cant start AdminServer it's failing complaining passwords are expired. (I could have get a way with this issue if i keep the admin server running and set the new passwords)
I can see DataSources are targeted to Admin Server and I thought if I untarget DS from AdminServer I could start AdminServer correctly. hence I removed the AdminServer as a target from config.xml and tried o start the Admin but it's still failing complaining passwords are expired. Is Config cached anywhere , looks like I am Admin is still using the old config file ? by the way I have tried removing the tmp folder as well.
Also, I tried encrypting the new password and placing on JDBC config files, Probably the way I encrypted was wrong. These are the steps I used to encrypt
1. Connect to WLST offline ( because Admin is not up)
2. Read domain
3. Call encrypt function for new password
4. Print the encrypt password
Anything wrong ? Appreciate any suggestion to resolve this issue.
Error is starting like this ,
Jun 22, 2015 4:38:04 PM oracle.security.jps.JpsStartup start
INFO: Jps initializing.
Jun 22, 2015 4:38:07 PM org.hibernate.validator.util.Version <clinit>
INFO: Hibernate Validator 12.1.3.0.0
Jun 22, 2015 4:38:07 PM org.hibernate.validator.engine.resolver.DefaultTraversableResolver detectJPA
INFO: Instantiated an instance of org.hibernate.validator.engine.resolver.JPATraversableResolver.
[EL Severe]: ejb: 2015-06-22 16:38:11.173--ServerSession(143991231)--Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
Error Code: 1017
Jun 22, 2015 4:38:11 PM oracle.security.jps.internal.common.config.AbstractSecurityStore getSecurityStoreVersion
WARNING: Unable to get the Version from Store returning the default oracle.security.jps.service.policystore.PolicyStoreException: javax.persistence.PersistenceException: Exception [EclipseLink-4002] (Eclipse Persistence Services - 2.5.2.v20140319-9ad6abd): org.eclipse.persistence.exceptions.DatabaseException
Internal Exception: java.sql.SQLException: ORA-01017: invalid username/password; logon denied
Error Code: 1017
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.processJPAException(JpsDBDataManager.java:2180)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.init(JpsDBDataManager.java:1028)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.jpsObjectBaseQuery(JpsDBDataManager.java:3089)
at oracle.security.jps.internal.policystore.rdbms.JpsDBDataManager.queryBaseObjects(JpsDBDataManager.java:5761)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersion(AbstractSecurityStore.java:211)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.getSecurityStoreVersion(AbstractSecurityStore.java:195)
at oracle.security.jps.internal.common.config.AbstractSecurityStore.<init>(AbstractSecurityStore.java:99)
at oracle.security.jps.internal.credstore.AbstractCredentialStore.<init>(AbstractCredentialStore.java:104)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStore.<init>(LdapCredentialStore.java:130)
at oracle.security.jps.internal.credstore.ldap.LdapCredentialStoreProvider.getInstance(LdapCredentialStoreProvider.java:235)
at oracle.security.jps.internal.credstore.rdbms.DbmsCredentialStoreProvider.getInstance(DbmsCredentialStoreProvider.java:101)
at oracle.security.opss.internal.runtime.ServiceContextManagerImpl.createContextInternal(ServiceContextManagerImpl.java:432)
Thanks.
First take backup of complete config folder inside domain.It looks like you are using rdbms policy store inside domain.So check for security-realm tag in config.xml there you will able to find encrypted password change the same using newly encrypted password and your admin server should start.
I have been trying to follow the guidelines in this Microsoft article to authenticate
against Apache with Kerberos and AD. I have successfully tested the communication between the apache server and the AD server with kinit. However when I attempt to access a restricted page on the server with IE I get an Internal server error and the following appears in the apache error log.
[Wed Sep 24 14:18:15 2008] [debug] src/mod_auth_kerb.c(1483): [client 172.31.37.38] kerb_authenticate_user entered with user (NULL) and auth_type Kerberos
[Wed Sep 24 14:18:15 2008] [debug] src/mod_auth_kerb.c(1174): [client 172.31.37.38] Acquiring creds for HTTP/srvnfssol1.dev.local#DEV.LOCAL
[Wed Sep 24 14:18:15 2008] [error] [client 172.31.37.38] gss_acquire_cred() failed: Miscellaneous failure (see text) (Key table entry not found)
I have run a truss on the apache process and confirmed that it is in fact loading up the keytab file ok. I am wondering if there is something wrong with the format of the keytab file...
HTTP/srvnfssol1.dev.local#DEV.LOCAL
I am not sure what I am missing though. Or what other things to check.
Any suggestions?
Thanks
Peter
Ok. Keytabs are supposed to contain the Service principal name, in this case "HTTP/srvnfssol1.dev.local#DEV.LOCAL" and the encryption key. I see where the MS docs say just to echo that to a file, but I don't think that's right.
You'll need to use the ktpass utility to create the keytab. The MS docs are here.
In particular, you'll need to specify KRB5_NT_SRV_HST, and most of the rest of the options can be default.
Sample of it on my machine:
C:\>ktpass /out test.keytab /princ HTTP/srvnfssol1.dev.local#DEV.LOCAL
/ptype KRB5_NT_SRV_HST /pass *
Type the password for HTTP/srvnfssol1.dev.local:
Key created.
Output keytab to test.keytab:
Keytab version: 0x502
keysize 62 HTTP/srvnfssol1.dev.local#DEV.LOCAL
ptype 3 (KRB5_NT_SRV_HST) vno 1 etype 0x1 (DES-CBC-CRC)
keylength 8 (0xa7f1fb38041c199e)
If the active directory server is the KDC, you'll need to use the /map <name> argument, where <name> is the computer account in active directory representing the server.
Some details on how all this works. When you browse to the website it should respond with a WWW-Authenticate: Negotiate header, and your browser will send a request to the KDC (active directory server) to get a kerberos ticket for the service. The AD server will look up the encryption key for the ticket using the service principal name, and send an encrypted service ticket back to the browser. Once the browser has the service ticket, it'll reissue the HTTP request with an authenticate header containing the ticket. The apache server will look up its key in the keytab, decrypt the ticket, and grant access.
The "key table entry not found" error happens because apache isn't finding itself in the keytab. Can also happen if the name resolution/realms aren't set up right.
You should be able to see all the kerberos requests AP-REQ/AP-REP/TGS-REQ/TGS-REP using wireshark on the client, tcp or udp port 88.