Experimenting with Terraform and Ansible deployment. I have a TF template that stands up a VPC and ec2 instances in AWS, and uses cloud-init scripts to prepare the instances. I've tried to upload my own SSH keys from Puttygen in the cloud-init scripts but I am receiving a "key_load_public: invalid format" error when testing Ansible. My cloud init is as follows:
#Add Users
users:
- name: ansadmin
sudo: ["ALL=(ALL) NOPASSWD:ALL"]
groups: [ sudo ]
homedir: /home/ansadmin
shell: /bin/bash
passwd: $6$UwvMBQ9VAUiKDcx$/kIbPH2GxwBbpevZEJtIyjXZIW7KJgth3MXEmxhqNH.TH2yonVMz2Ob7ROD7jGKCagF3iYkYVBnkL.okPkP/d0
ssh-authorized-keys:
- ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCe5RaiG1mJ6QxN318GGY1ecihS61R0WssUceOnvetVeQmMdyn/H5+BFedb+RfNZwqWs0Bnj4i6fY9EDg7ylR5damZMMH+YZD4bkTkzwbjWiFudlWZYcGKorvg8mXUoI0tE9gJOtbTS2sv7M0jG3YnqEc3zY9sDTPxyjSWHXLnjF505Zk61rvXnc5fRkUKrSdrvzl3I+0kCWZ0dpQ0PSrDSVEiNR/5f5HhCaq2a16i7kW28amzdcUduX4h8/wwt83b+YxK9kV73LGsPTyB79LHVCL9QUt4ktnqU/yHQ2ZJwlsJtOIZRc5L2aPZBc+8OuEN/i5H28TbyBF1mvFezjfXt
ssh_keys:
rsa_private: |
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAvV1UuXGnfWvoxoameuCuoUx1yPUeZeORovbzge0SpFctBs0k
Nqp37h8CG8wFADPXxK3xAMDunrY+HVPzQaVDeRlDIF5sRtb0vHS3NN9M6jmQb6W1
l/O3O1Brrp6oIEUxb1JOgCqHMw+48VFOmcdpvGZ4hf7a8w+OUK6XUGhC1A/nv6Dx
4Icv+KhX5OwxrytN7oCyHdQiQpPfB8fMtX31+lE1YBvwtcSJaNc/dOP9ONTYXPwe
fJYenapMTzewl1q+OkImf8DTgz9gmBMxydZdrwutOFrxpzkcVX0KITIyH1r0hTzY
1LEG1z/8qhxijfu49lYZ2YvB2TnlbiCN/9O6+QIDAQABAoIBADbvzbWjIGlFlhmK
VOAPMPqAmSc83Z3HoKe7pzeJVMAyvlBb0Wa+m96MRmtUDBtb7vwTTnjegBuxWWHe
yhLAZEvMpWnNmvIOSKAn6ELBianmG14YoA8+WGkv0p2tT69VA5t9MGMEc4a/x/LS
mNAKR0WuIMcXAeREHhCQJjvgk4q7trbOrKYgKxJsMRFwJXhYW95ZJSXjKuAVj8se
0B5BnVWTFj3cxhgf7DVVuXrb83zIBK+6Ucq4X+nMGy8pLBw+jmcgb/Y3ssXxnh6b
uijvUuc/i4yI9R4OwfV5tkpMFsbh2y74OS2UDfCtYCwiKjOGKEfv1oukfDSlCaqs
2DjgacUCgYEA4/DOCfJVGWmVGsVxsJjg5k8nt/6FdJ4zx7jkvFjZ7UB6QfCmznLs
3BRkX1ZrurRazMYNq56FXT8K6JPHsttwSy6zj/KYRRFhdEUo6EHJSELVZZHr9qIk
duE0Fz+9FddY2jno8jyxA1+tTYudV2lkp1cVVb/UvGQCXSadmJrt+x8CgYEA1Kzc
eRtpbKf+jPxYylqm7gPp3I3K5CgGa1gdjOTS8yc/DBbqPCQkrQe6pMKAaiWjwHz4
y77yYmp1JtsPip7kEm2pSXVfDBmdj+aVBRUm+mDa/lXC1BgG3zaz8MsnuqoMR+19
BO6PXN/yAEau5EnlNK6rD8A3VvvHDzXu/EPHnucCgYEAgLOa4qrdufePpQoO43Ou
qNvQxWcE24/oFMElBeP1SKy7WzgSN0dUzf8FnX8iXM0w56Z8WYasLrZF5oRqoWad
xRWddY24xGbH6+qQ6nMav55c93ipDx0GEcmeq/HlRcvN84n3Ka9zE0CWjc5jDNQJ
bSMSfNRPvf+KBiajnWL9NGkCgYA2d06/dVW6jguS782NhUnbCwWJhoa/h8CdHSP0
QWYE/7gV0IiMfnUmGyShrW5VwO3/DgJpq61Hpxv5p4CDb83ZKlyAg56j6qt/fv4L
Hy+sT8HGARC0YLLh15CdymmSVJpwkVHDQZVVo8TGbVO9A2+/3jvQ/NkvXavNKXLb
CEDHYQKBgQDJczoRCSmIQnloCFonNmCaWIGC7/dAjHU6VeNpbxyUGVqTiMBhG7Mk
XIR454e1RYrielxGlPJZYRmIcK1SmI0sb/obkqbr7qLxrIMrcFqHQLatLlu6rd2K
Url/rg6af1DzvYIrMGop4vbIZhBPeP0iyB9GGxIyE+Hh9VTU1MYEYQ==
-----END RSA PRIVATE KEY-----
rsa_public: ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCe5RaiG1mJ6QxN318GGY1ecihS61R0WssUceOnvetVeQmMdyn/H5+BFedb+RfNZwqWs0Bnj4i6fY9EDg7ylR5damZMMH+YZD4bkTkzwbjWiFudlWZYcGKorvg8mXUoI0tE9gJOtbTS2sv7M0jG3YnqEc3zY9sDTPxyjSWHXLnjF505Zk61rvXnc5fRkUKrSdrvzl3I+0kCWZ0dpQ0PSrDSVEiNR/5f5HhCaq2a16i7kW28amzdcUduX4h8/wwt83b+YxK9kV73LGsPTyB79LHVCL9QUt4ktnqU/yHQ2ZJwlsJtOIZRc5L2aPZBc+8OuEN/i5H28TbyBF1mvFezjfXt
write_files:
- content: |
ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAABAQCe5RaiG1mJ6QxN318GGY1ecihS
61R0WssUceOnvetVeQmMdyn/H5+BFedb+RfNZwqWs0Bnj4i6fY9EDg7ylR5damZM
MH+YZD4bkTkzwbjWiFudlWZYcGKorvg8mXUoI0tE9gJOtbTS2sv7M0jG3YnqEc3z
Y9sDTPxyjSWHXLnjF505Zk61rvXnc5fRkUKrSdrvzl3I+0kCWZ0dpQ0PSrDSVEiN
R/5f5HhCaq2a16i7kW28amzdcUduX4h8/wwt83b+YxK9kV73LGsPTyB79LHVCL9Q
Ut4ktnqU/yHQ2ZJwlsJtOIZRc5L2aPZBc+8OuEN/i5H28TbyBF1mvFezjfXt
path: /home/ansadmin/.ssh/id_rsa.pub
permissions: '0600'
owner: ansadmin:ansadmin
defer: true
- content: |
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAvV1UuXGnfWvoxoameuCuoUx1yPUeZeORovbzge0SpFctBs0k
Nqp37h8CG8wFADPXxK3xAMDunrY+HVPzQaVDeRlDIF5sRtb0vHS3NN9M6jmQb6W1
l/O3O1Brrp6oIEUxb1JOgCqHMw+48VFOmcdpvGZ4hf7a8w+OUK6XUGhC1A/nv6Dx
4Icv+KhX5OwxrytN7oCyHdQiQpPfB8fMtX31+lE1YBvwtcSJaNc/dOP9ONTYXPwe
fJYenapMTzewl1q+OkImf8DTgz9gmBMxydZdrwutOFrxpzkcVX0KITIyH1r0hTzY
1LEG1z/8qhxijfu49lYZ2YvB2TnlbiCN/9O6+QIDAQABAoIBADbvzbWjIGlFlhmK
VOAPMPqAmSc83Z3HoKe7pzeJVMAyvlBb0Wa+m96MRmtUDBtb7vwTTnjegBuxWWHe
yhLAZEvMpWnNmvIOSKAn6ELBianmG14YoA8+WGkv0p2tT69VA5t9MGMEc4a/x/LS
mNAKR0WuIMcXAeREHhCQJjvgk4q7trbOrKYgKxJsMRFwJXhYW95ZJSXjKuAVj8se
0B5BnVWTFj3cxhgf7DVVuXrb83zIBK+6Ucq4X+nMGy8pLBw+jmcgb/Y3ssXxnh6b
uijvUuc/i4yI9R4OwfV5tkpMFsbh2y74OS2UDfCtYCwiKjOGKEfv1oukfDSlCaqs
2DjgacUCgYEA4/DOCfJVGWmVGsVxsJjg5k8nt/6FdJ4zx7jkvFjZ7UB6QfCmznLs
3BRkX1ZrurRazMYNq56FXT8K6JPHsttwSy6zj/KYRRFhdEUo6EHJSELVZZHr9qIk
duE0Fz+9FddY2jno8jyxA1+tTYudV2lkp1cVVb/UvGQCXSadmJrt+x8CgYEA1Kzc
eRtpbKf+jPxYylqm7gPp3I3K5CgGa1gdjOTS8yc/DBbqPCQkrQe6pMKAaiWjwHz4
y77yYmp1JtsPip7kEm2pSXVfDBmdj+aVBRUm+mDa/lXC1BgG3zaz8MsnuqoMR+19
BO6PXN/yAEau5EnlNK6rD8A3VvvHDzXu/EPHnucCgYEAgLOa4qrdufePpQoO43Ou
qNvQxWcE24/oFMElBeP1SKy7WzgSN0dUzf8FnX8iXM0w56Z8WYasLrZF5oRqoWad
xRWddY24xGbH6+qQ6nMav55c93ipDx0GEcmeq/HlRcvN84n3Ka9zE0CWjc5jDNQJ
bSMSfNRPvf+KBiajnWL9NGkCgYA2d06/dVW6jguS782NhUnbCwWJhoa/h8CdHSP0
QWYE/7gV0IiMfnUmGyShrW5VwO3/DgJpq61Hpxv5p4CDb83ZKlyAg56j6qt/fv4L
Hy+sT8HGARC0YLLh15CdymmSVJpwkVHDQZVVo8TGbVO9A2+/3jvQ/NkvXavNKXLb
CEDHYQKBgQDJczoRCSmIQnloCFonNmCaWIGC7/dAjHU6VeNpbxyUGVqTiMBhG7Mk
XIR454e1RYrielxGlPJZYRmIcK1SmI0sb/obkqbr7qLxrIMrcFqHQLatLlu6rd2K
Url/rg6af1DzvYIrMGop4vbIZhBPeP0iyB9GGxIyE+Hh9VTU1MYEYQ==
-----END RSA PRIVATE KEY-----
path: /home/ansadmin/.ssh/id_rsa
permissions: '0600'
owner: ansadmin:ansadmin
defer: true
- content: |
ansadmin ALL=(ALL) NOPASSWD: ALL
path: /etc/sudoers
append: true
If I replace the key using "ssh-keygen", I can use "ssh-copy-id" to the client node with no errors - so I presume something must be wrong with my format coming from Puttygen. The literal format - how the Puttygen public key is laid out in id_rsa.pub - looks exactly the same as the key produced by ssh-keygen. Any thoughts?
I solved it on my own by just copying a key directly from ssh-keygen instead. Not sure why the Puttygen key didn't work, but whatever works!
Related
In other words, I want the content from an Ansible variable to be written to a particular "column" in a config file, and I don't want that content to be wrapped.
The task:
- name: build cloud-init file
local_action:
module: ansible.builtin.template
src: cloud_config.j2
dest: ./cloud_config.yml
mode: 0640
tags: ['containers', 'containers:configuration']
The variable, which lives in a vault file:
machineuser_key: |
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
...
The template:
write_files:
{# ssh config to allow our deployment user to fetch the Ansible repo #}
{# See https://docs.github.com/en/developers/overview/managing-deploy-keys#machine-users #}
- path: /home/{{ gcp_deploy_user }}/.ssh/{{ gcp_deploy_user}}_github_rsa
permissions: 600
owner: {{ gcp_deploy_user }}
content: |
{{ machineuser_key }}
The output of that template task:
write_files:
- path: /home/omegasphinx/.ssh/omegasphinx_github_rsa
permissions: 600
owner: omegasphinx
content: |
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
...
I know that blockinfile is a recommended module when one wants to write a config file, and that I should use the "Block Indication Indicator" to do this. But when I do these things, it still wraps the content of the ssh key. That blockinfile task looks like this:
- name: add ssh private key to cloud-init file
local_action:
module: ansible.builtin.blockinfile
path: cloud_config.yml
mode: 0440
insertafter: "# machineuser_ssh_block"
block: |2
- path: /home/{{ gcp_deploy_user }}/.ssh/{{ gcp_deploy_user}}_github_rsa
permissions: 600
owner: {{ gcp_deploy_user }}
content: |
{{ machineuser_key }}
tags: ['containers', 'containers:configuration']
And the .j2 template looks like this:
write_files:
# machineuser_ssh_block
The output then looks like this (exactly the same):
write_files:
- path: /home/omegasphinx/.ssh/omegasphinx_github_rsa
permissions: 600
owner: omegasphinx
content: |
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
...
Other things I've tried:
I disable tabs-to-spaces in my editor and then added tabs before each line of the ssh key in the machineuser_key variable
The docs say "You can manually disable the lstrip_blocks behavior by putting a plus sign (+) at the start of a block"; so I added a block and then indented the variable inside the block:
{%+ if machineuser_key is defined %}
{{ machineuser_key }}
{% endif %}
My understanding of whitespace in Jinja2:
To give you some context on what I'm expecting to happen. The "Whitespace Control" section of the Jinja docs states that "a single trailing newline is stripped if present" but "other whitespace (spaces, tabs, newlines etc.) is returned unchanged." In my case, I have "other whitespace": spaces to the left of each line of the variable machineuser_key. But obviously I'm missing something.
As far as I can tell from your question, it's not jinja whitespace control that's jamming up your outcome, it's the fact that machineuser_key itself has embedded newlines; a lot of helm charts have the same problem, which allows you to take advantage of the same work-arounds:
take advantage of the fact that YAML is a superset of JSON
use the indent filter to get you and the YAML block on the same page
JSON as YAML approach
- path: /home/{{ gcp_deploy_user }}/.ssh/{{ gcp_deploy_user}}_github_rsa
permissions: 600
owner: {{ gcp_deploy_user }}
content: {{ machineuser_key | to_json }}
resulting in
owner: gcpawesomeuser
content: "-----BEGIN OPENSSH PRIVATE KEY-----\nb3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn\nETC ETC ETC\n"
Using the indent filter
- path: /home/{{ gcp_deploy_user }}/.ssh/{{ gcp_deploy_user}}_github_rsa
permissions: 600
owner: {{ gcp_deploy_user }}
content: |
{{ machineuser_key | indent(2) }}
# with the (2) matching the number of spaces under "content" where the mustaches start
producing:
owner: gcpawesomeuser
content: |
-----BEGIN OPENSSH PRIVATE KEY-----
b3BlbnNzaC1rZXktdjEAAAAABG5vbmUAAAAEbm9uZQAAAAAAAAABAAABlwAAAAdzc2gtcn
ETC ETC ETC
Note: I will regenerate all passwords and certs in my final install since I have provided all that information here. Also I have asked the same question here
https://discuss.elastic.co/t/kibana-ssl-pem-error-on-windows/117851/2
Environment
Windows 10 Enterprises
Elasticsearch Version : 6.1.1 (elasticsearch-6.1.1.zip)
Kibana Version: 6.1.1 (kibana-6.1.1-windows-x86_64.zip)
Problem:
I am trying to enabled SSL in kibana and I am running into an issue. I cant for the life of me figure out what I have done wrong. Here are the steps I have taken so far.
0) Create Yaml file use with the certutil --in flag.
instances:
- name: node1
dns: ['node1.local']
- name: devws-kibana
dns: ['devws-kibana.local']
1) Declare variables for use in powershell commands
$root = "C:\working\elasticsearch"
[Version]$esVersion = "6.1.1"
$es = "$root\elasticsearch-$($esVersion.ToString())"
$esService = "elasticsearch_$($esVersion.ToString() -replace '\.','')"
[Version]$KibanaVersion = "6.1.1"
$kibana = "$root\kibana-$($KibanaVersion.ToString())-windows-x86_64"
$kibanaService = "elasticsearch-kibana$($KibanaVersion.ToString() -replace '\.','')"
2) Install x-pack in elasticsearch
`&"$es\bin\elasticsearch-plugin.bat" install x-pack --batch`
3) Create a Self Signed CA certificate.
&"$es\bin\x-pack\certutil.bat" ca --silent --pass password --ca-dn "CN=Elasticsearch-DevWS" --pem --out "$root\elastic-stack-ca.zip"
Expand-Archive -Path "$root\elastic-stack-ca.zip" -OutputPath "$root\certs"
4) Create a cert for elasticsearch and kibana
&"$es\bin\x-pack\certutil.bat" cert --silent --pem --ca-cert "$root\certs\ca\ca.crt" --ca-key "$root\certs\ca\ca.key" -in "$root\instances.yml" --ca-pass password --pass password --out "$root\certificate-bundle.zip"
Expand-Archive -Path "$root\certificate-bundle.zip" -OutputPath "$root\certs"
5) Copy Certs to proper directories
Copy-Item -Path "$root\certs\ca\ca.crt" -Destination "$es\config\certs\ca.crt"
Copy-Item -Path "$root\certs\node1\*" -Destination "$es\config\certs\"
Copy-Item -Path "$root\certs\ca\ca.crt" -Destination "$kibana\config\certs\ca.crt"
Copy-Item -Path "$root\certs\devws-kibana\*" -Destination "$kibana\config\certs\"
6) Update Elasticsearch.yml to below
cluster.name: WRK001
node.name: node1
network.host: node1.local
http.port: 9210
discovery.zen.ping.unicast.hosts: [ 'node1.local' ]
processors: 2
node.master: true
node.data: true
node.max_local_storage_nodes: 1
xpack.ssl.key: certs/node1.key
xpack.ssl.certificate: certs/node1.crt
xpack.ssl.certificate_authorities: certs/ca.crt
xpack.security.transport.ssl.enabled: true
xpack.security.transport.ssl.verification_mode: full
xpack.security.http.ssl.enabled: true
xpack.ssl.key_passphrase: password
7) Add secure key passphrase to keystore
"password" | &"$es\bin\elasticsearch-keystore.bat" add xpack.ssl.secure_key_passphrase --stdin
&"$es\bin\elasticsearch-keystore.bat" list
8) Start Elasticsearch
9) Set password for build in accounts
$url = https://node1.local:9210/
$output = & cmd.exe /C "$es\bin\x-pack\setup-passwords.bat auto --url $url -batch" 2>&1
Write-Host -ForegroundColor Green -BackgroundColor Black $output
10) Parse passwords from response and save to temp files.
$elasticPassword = ($output | Select-String -Pattern "^PASSWORD\selastic\s=\s(.*)$" -AllMatches).Matches[0].Groups[1].Value
$kibanaPassword = ($output | Select-String -Pattern "^PASSWORD\skibana\s=\s(.*)$" -AllMatches).Matches[0].Groups[1].Value
$elasticPassword | Out-File -FilePath "$es\config\elastic.password" -Encoding utf8
$kibanaPassword | Out-File -FilePath "$kibana\config\kibana.password" -Encoding utf8
11) Remove setting xpack.ssl.key_passphrase from Elasticsearch.yml
12) Restart Elasticsearch
13) Verify Elasticsearch is work (and it is)
14) Install X-Pack in kibana
&"$kibana\bin\kibana-plugin.bat" install x-pack
15) Update Kibana.yml to below
server.name: devws-kibana
server.host: devws-kibana.local
elasticsearch.url: https://node1.local:9210/
elasticsearch.username: kibana
elasticsearch.password: nWD0zPDLFiM3yHdVQM9j
elasticsearch.ssl.certificateAuthorities: ../config/certs/ca.crt
16) Start Kibana
17) Verify Kibana is running and I am able to log in with elastic user
18) Stop Kibana
19) Update Kibana.yml to below
server.name: devws-kibana
server.host: devws-kibana.local
server.ssl.enabled: true
server.ssl.certificate: ../config/certs/devws-kibana.key
server.ssl.key: ../config/certs/devws-kibana.crt
elasticsearch.url: https://node1.local:9210/
elasticsearch.username: kibana
elasticsearch.password: nWD0zPDLFiM3yHdVQM9j
elasticsearch.ssl.certificateAuthorities: ../config/certs/ca.crt
xpack.security.encryptionKey: 3qrb1xee9ue9rrh3p93ykj28otgp676iu0l8ziifjopfov6h4sv9jhyp49gpm90t
20) Try starting kibana. It fails and produces the following error.
FATAL Error: error:0906D06C:PEM routines:PEM_read_bio:no start line
at Error (native)
at Object.createSecureContext (_tls_common.js:69:17)
at Server (_tls_wrap.js:776:25)
at new Server (https.js:26:14)
at Object.exports.createServer (https.js:47:10)
at new module.exports.internals.Connection (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\node_modules\hapi\lib\connection.js:88:74)
at internals.Server.connection (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\node_modules\hapi\lib\server.js:142:24)
at KbnServer.exports.default (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\http\setup_connection.js:43:10)
at C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\kbn_server.js:171:20
at next (native)
at step (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\kbn_server.js:87:191)
at C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\kbn_server.js:87:437
at C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\kbn_server.js:87:99
at KbnServer.mixin (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\kbn_server.js:187:7)
at KbnServer.<anonymous> (C:\working\elasticsearch\kibana-6.1.1-windows-x86_64\src\server\http\index.js:66:21)
at next (native)
Documentation:
https://www.elastic.co/blog/tls-elastic-stack-elasticsearch-kibana-logstash-filebeat
https://www.elastic.co/guide/en/kibana/6.1/installing-xpack-kb.html
https://www.elastic.co/guide/en/kibana/6.1/security-settings-kb.html
https://www.elastic.co/guide/en/kibana/current/production.html#enabling-ssl
https://www.elastic.co/guide/en/kibana/6.1/using-kibana-with-security.html
https://www.elastic.co/guide/en/kibana/6.1/settings.html
https://www.elastic.co/guide/en/elasticsearch/reference/6.1/certutil.html
devws-kibana.key
-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,17D5A5CA90BECF38
zOhDSJ8OFqB+FGvSmL18k4kIiojvO4yLnAmTYDar51NGAdvitBlGaOEvXzpBAx9j
3nwNVBvz+NCBMg9tOcr8UY9H1/Qgns7gpiWhCJmJlja33Jly97UJS9go6DHvWaJi
OoppKOHOkFaDDnMo6XkyHN2PXagVxFJxO8zW1OgQjMn1dWzUN8jwff/hUmgLT1tx
Arsso/+OKWEwZIJ2rczJjxXjgQmAu+d3libn8tLG9TpDStnj47j/8nxyViIp8Rq0
fFv3XW2QN8oj0Pjo0IBP8FjjGc8+b+zq+kgfvs0Lz65k5GigUcL66nkAgXTADb0S
1Qxo8NVgC4yzc94WrMqSiV99Tlw1kCh1Sb86TJDjYvWMko04wP+kFJrIPs2IxA9D
qxf7YwSZUHfIxg9FdNJHa+jHy+HinX9wOLK6JxZv5jWcZDNThjUW9JxRdvuM1nxD
6K0bUWAa8VlO0MU9x8Hwo7hhT4lJp6sr4jagKkvEqo/ZJWqyQzv5sOMQKu8vXGco
nqjxApjTQGIplf+RrYaPg7URH6GdOW3cIY0d+Hh9i1K/v7UolO2fZjSoCaYmt4YF
BjsMTIpdMnw1GfT0TlRa3bTgRAIqI9lE008fS/XRhjHgKY6wgHbbb7L+mxG05FIa
8rOAClu1btBJXZSELK4JqUgbY+kIPQqB2Ezh1FpggjTboJkdF2cXCazCLDtMSEi3
z95miNGm3rRtS0HNaqrElDQeNbbyo6X9joFKApaxvdICDe9rfdMbhl/tQkYZdyxA
XKeqbPL1dS95ma60A/7B0fIqwrD8Rko8q0EIMClG7bC/C7JRSdd8YEpIV9B1xorc
gYyPiLtoGsihFpF2wpapG0e45wKedtl5QAYbeMWp9JoRPx9ZUuaP4Usn6zdyCqtw
mWUGAHw2F7qQYpdvvUYrKAwC4dVRwivaHz4OoabppY0dYCq1THGvFBmlinbtDFLD
ihKwgnvdACyhWU+e45a53cAtpSwoaPVenFvUJU6MHt+XmxT757blb4Dn9nsHqrwz
HyWo0QoXdgbCwAzzBRW90R6wwYJx2cD+aYxpFop7aeeM/BgQ7IK2ejnykaTk3+NO
hTJ1RgUDvn30jmczvJE0hquzn4SxKVi/hn+YuPxENc2a3CDbj6vsLQRGYMxjnSwT
4POYxc+hgszaE7afh2sKjQHwNboHpeLNuFOXDU22StsIbltWltw+jKzz3sHNE83G
uUJnJPi7LN47WOT2pTNqV+k7/SEFfdo4JVDna54LF8LkrNLl9odmXR3ryTuTdQwP
fAiNXWySIKfEJufVb4g0tv3xDFtJztwZrVvOgCnMnlXw+4BWp7FmN3M+pjDReEdv
1t15tPoDltCfPRva4bZaUXriYSYK5w6PW0t33AR7WbUSKmYlisqInCXwACjBjQJc
nN8epVm33Kaqs18T1PSfkUKiqCKfFRqgleL5W96lG/W8b6qZvglod1qIu3k7qqiy
V6d/jLpe/lASCy1mCtjnNMYTkIlNc5Jyy7FrplXGEm5dWLunMCnCAYX4rfAunynW
1YdB/poa9pXBbUPrF1KxHVNAmV7x5VS4/0eK7LKcUAWs01nH7yiBsXEfs3YfsVMQ
-----END RSA PRIVATE KEY-----
devws-kibana.crt
-----BEGIN CERTIFICATE-----
MIIDbDCCAlSgAwIBAgIUMtq8dgtI+Opfy++z6Ptx8ci2Pr8wDQYJKoZIhvcNAQEL
BQAwHjEcMBoGA1UEAxMTRWxhc3RpY3NlYXJjaC1EZXZXUzAeFw0xODAxMzAxOTUz
NTNaFw0yMTAxMjkxOTUzNTNaMBcxFTATBgNVBAMTDGRldndzLWtpYmFuYTCCASIw
DQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKxUb4Mc9PVov9+r6a7V1At1+JL/
CXl81g3tFQNYLzAMj/RJr06HAht8Loh7i7rBvWjleklOZNoKC8LMhbDNU56ix4WK
46yboRknYbjyEYOTjujx6gKoZ72yHugcbF5mc6GoafnyDJf0zL5nNHD8rC61WsFg
hUzJF0qlA9dMojjMMrlrDJDIeKRzhrDj950QBgC9bzLVHQ0sseHmmPQFRQRXXYiP
7/89jfrclgsRaB/H7A2DI5JnZz/HvTQ7UiVOoRXb4TW3NMc1OQbkTg2bjpPMruL6
tVXE/F0xLL4XXSUKbzT95p3vGxRp0jbnce0c8u4USiBEWoJbmYmD/rOouOECAwEA
AaOBqDCBpTAdBgNVHQ4EFgQUpbjtvSNDqoOha32Oculkz4L9c6kwWgYDVR0jBFMw
UYAU3G6UbG/SC9ucNlVGjGZmwk/CQaehIqQgMB4xHDAaBgNVBAMTE0VsYXN0aWNz
ZWFyY2gtRGV2V1OCFQDW6SMoh4MojBt5XyEkdFAA85u/jTAdBgNVHREEFjAUghJk
ZXZ3cy1raWJhbmEubG9jYWwwCQYDVR0TBAIwADANBgkqhkiG9w0BAQsFAAOCAQEA
Nr0wX2LqbIV8c4/tXaPUA34J8WoWS0HarADrFA+NFee3+/V0Afoun20HI8B8ql6a
5sK+omjDTsHxjjGIitatS/ntmOacJST4MLVeQzSHEfx3ArvozD0LJqnpCifvvUMz
0Gq1fmZHUioz3sHeqw4ZlpEll8MRyvtVj6gDQqzUJcaw9S06/J3VsEJbrRudhSLD
GYgCz1P5jIsZoiEFhinWafgayImcYY3LMNdpRFwIbmwO4Uhs26vgt2zfRVW3vw6r
+5O+DMvjoSe46fZN4uY36rCAEiCoYDC+PB6LSFOGbGswylTTL6F8EorQtjJxOnsn
yFmugwITE4K4iwAQcuCy+g==
-----END CERTIFICATE-----
ca.crt
-----BEGIN CERTIFICATE-----
MIIDWzCCAkOgAwIBAgIVANbpIyiHgyiMG3lfISR0UADzm7+NMA0GCSqGSIb3DQEB
CwUAMB4xHDAaBgNVBAMTE0VsYXN0aWNzZWFyY2gtRGV2V1MwHhcNMTgwMTMwMTk1
MzQ5WhcNMjEwMTI5MTk1MzQ5WjAeMRwwGgYDVQQDExNFbGFzdGljc2VhcmNoLURl
dldTMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEArB0kt1hdL0xuCqtw
pO/9wr6xvL9uOy63uHWhEvF8hqW1Kv3w9unZQkJxXlQj3oVbhJTGD6+Bu6RZ8nwl
J7kVjf3EHDvcvwZZElHjmV0zlZ8k3XlJJmKIFeCNAa52YpmReiFerIv+xWV9F4Ae
B77O2pNzfvrJzWroPBVodbF9/N0kxplwSbAJPRGLDvknxW0vX3XiyjvDUPZkmVhm
xc7g0XkTqtjGcYKylz3sfCEnBOSY+3TKePyA62thKlmfMb5iDxGHjraHCcXzPtjh
y3LcD4E3KM57xv1XnHyrKxzJLf0iaJb1xyd4aRGFfckhkqrGvyaS08PRLd3RL+QE
/JsRKQIDAQABo4GPMIGMMB0GA1UdDgQWBBTcbpRsb9IL25w2VUaMZmbCT8JBpzBa
BgNVHSMEUzBRgBTcbpRsb9IL25w2VUaMZmbCT8JBp6EipCAwHjEcMBoGA1UEAxMT
RWxhc3RpY3NlYXJjaC1EZXZXU4IVANbpIyiHgyiMG3lfISR0UADzm7+NMA8GA1Ud
EwEB/wQFMAMBAf8wDQYJKoZIhvcNAQELBQADggEBAIA7S3HbOEKw9kTjxPYlOYoe
kQiKTyZ7rxUAJYSnafnmsjbCMbpVXS9k+THm3IUTQuyxwrGuuBBeKFZJWQ1FcPiF
DVPsgsNO4MRT6r78XjmCJJZcB5FZqbfK7EQd/E4sRzq8bk5VR6wfQK/U5/42TTcw
5RdDYnS4axLQOb9AuSdma7XP6BcshNAFCTp39caP7ZfKLJeRMMv0Mn0/3Yt9I9dv
2MGpxnMOYeVKzYVeoyXXDIOZqdPEkPO6gO7i1MprHcC3XlXFwkbe/EZ4pKUtRTJU
kUgoSTOEd8BO8hwOYhG3HjOqTQe4U6lp2J58Kk47MMs8KUH5Zv47O8baNdHPWVw=
-----END CERTIFICATE-----
Figured out the issue. The server.ssl.certificate and server.ssl.key values were switched.
I am trying to use logstash http_poller to query a server RESTAPI. I download the server pem through explore, and generate jks file with keytool. but we still get error "PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target". Don't know what wrong.
The config like below:
http_poller {
urls => {
restapi => {
method => get
url => "https://path_to_resources
headers => {
Accept => "application/json"
}
truststore => "/path/generated.truststore.jks"
truststore_password => "xxx"
ssl_certificate_validation => false
auth => {
user => "xxx"
password => "xxx"
}
}
}
request_timeout => 60
interval => 60000
codec => "json"
metadata_target => "http_poller_metadata"
}
}
By the way, what impact if ssl_certificate_validation is set as false?
I interpret OPs intention as to hopefully being able to disable TLS verification, which we still cant (logstash-7.11.1) and I plow on with how to get a trust store for these cases. This Q was one of my hits in pursuit of the same.
Some appliances will be running self signed certificates (another discussion ppl...) - so a small script to setup such a trust store could be helpful, especially if you are about to set up some automation internally.
Another caveat is that the self signed certificate still has to have a matching host name.
Based on the example from https://www.elastic.co/guide/en/logstash/current/plugins-inputs-http_poller.html
NB! Further error checking, etc. is left at your discretion.
#!/bin/bash
# Fetch an http server's TLS certificate and
# create or update a JAVA keystore / truststore
usage () {
echo "usage: get-cert.sh <hostname>:<port>"
exit 1
}
TRUSTSTORE=cacert/trust.jks
PARAM=$1
HOSTNAME=$(echo "$PARAM" | cut -d: -f 1)
PORT=$(echo "$PARAM" | cut -d: -f 2)
REST=$(echo "$PARAM" | cut -d: -f 3-)
[ -z "$HOSTNAME" ] && usage
[ -z "$PORT" ] && usage
[ -n "$REST" ] && usage
OUTPUT=$(
openssl \
s_client \
-showcerts \
-connect "${HOSTNAME}":"${PORT}" </dev/null 2>/dev/null | \
openssl \
x509 \
-outform PEM)
EC=$?
[ $EC -ne 0 ] && { echo "ERROR EC=$EC - $OUTPUT" ; exit $EC ; }
keytool \
-import \
-storepass changeit \
-alias ${HOSTNAME} \
-noprompt \
-file <(echo "$OUTPUT") \
-keystore ${TRUSTSTORE}
Using some bash specific possibilities here. The alternative is to go through temporary files, as pr the official example (see link above).
Apparently your certificate is invalid .
Regarding
ssl_certificate_validation
it doesn't have real impact , http-puller is based on manticore, a ruby libary which relay on Apache HC
which does not support this hook see
On my puppet master I have a site.pp file that defines the content of some certificates like so ...
$swift_cert_content = '
-----BEGIN CERTIFICATE-----
blhablhadfblhablah...
-----END RSA PRIVATE KEY-----
'
... and I want to write the value of $swift_cert_content to the file /tmp/swift_cert on my puppet master. I have tried this ...
# cat create_cert_files.pp
class create_cert_files (
$swift_cert_content = $::swift_cert_content,
) {
file { '/tmp/swift_cert':
ensure => "file",
content => $swift_cert_content,
}
}
class { 'create_cert_files': }
... and I execute this like so ...
# puppet apply create_cert_files.pp
Notice: Compiled catalog for osc-ppt-001.example.com in environment production in 0.16 seconds
Notice: /Stage[main]/Create_cert_files/File[/tmp/swift_cert]/ensure: created
Notice: Finished catalog run in 0.31 seconds
... but the file is empty:
# ls -l /tmp/swift_cert
-rw-r--r-- 1 root root 0 Aug 26 14:55 /tmp/swift_cert
What am I doing wrong?
I forgot that when one uses puppet apply puppet ignores the site.pp file so I need to do something like this:
# cat /etc/puppetlabs/puppet/manifests/site.pp create_cert_files.pp > runit.pp
# puppet apply runit.pp
I create a db with certutil
myproject/bin/pkcert
echo "dartdart" > pwdfile
certutil -N -d 'sql:./' -f pwdfile
Then, I import my certificate validated by an authority
certutil -d "sql:./" -A -t "C,C,C" -n "my_cert" -i certificate.crt
I check if it work
certutil -L -d 'sql:./'
Certificate Nickname Trust Attributes
SSL,S/MIME,JAR/XPI
my_cert C,C,C
My main.dart
library main;
import "dart:io";
void main() {
var testPkcertDatabase = Platform.script.resolve('./pkcert')
.toFilePath();
SecureSocket.initialize(database: testPkcertDatabase,
password: 'dartdart');
HttpServer
.bindSecure(InternetAddress.ANY_IP_V6,
8443,
certificateName: 'my_cert')
.then((server) {
server.listen((HttpRequest request) {
request.response.write('Hello, world!');
request.response.close();
});
});
}
I execute, and I get this error:
Uncaught Error: CertificateException: Cannot find server certificate by nickname: my_cert (OS Error: security library: read-only database., errno = -8126)
I missed something or I should report a bug?
Thank you.
Have you tried to add -s "cn=mycert" to
certutil -d "sql:./" -A -t "C,C,C" -n "my_cert" -i certificate.crt
and use it with
HttpServer
.bindSecure(InternetAddress.ANY_IP_V6,
8443,
certificateName: 'CN=my_cert')
as shown in this blog post http://jamesslocum.com/post/70003236123 ?