How can I manage puppet agent nodes in the internal network from a puppet master in the internet?
All the puppet agent nodes are within an internal network, thus do not have a unique ip for each of the agent node, however the puppet master node is in the internet, how can accomplish this? Any help will be greatly appreciated.
Node's IP address doesn't participate in the authentication process to the puppet master. Also, master does not need to directly connect to nodes, since it's nodes responsibility to contact the master which then provides compiled catalog for each node.
What's important is that node's FQDN is unique among all the nodes managed by master. As far as:
Puppet master can be reached via a known FQDN (by default configured to puppet on the agents, might be changed to a full domain name mapped to a public IP in your case)
Puppet agents have access to Internet (or, more specifically, at least to the IP address of the Puppet master)
Puppet agents have a unique fully qualified domain name configured on them (which you can see with hostname -f)
your master can manage your agents without special configuration.
Related
I'm specifically trying to do this with Apache Storm (1.0.2), but it's relevant to any service that is secured with Kerberos. I'm trying to run a secured Storm cluster in Docker. There are a number of out-of-the-box docker images out there for Storm, and they work great unsecured. I'm using https://github.com/Baqend/docker-storm. I also have Storm running securely on RHEL VM's.
However, my understanding is that Kerberos ties hostnames to principals, so if I'm making service foobar available to clients, I need to create a principal of foobar/hostname#REALM. Then a client service might connect to hostname with principal foobar, Kerberos will look up foobar/hostname#REALM in its database, find that it's there (because we created a principal with exactly that name), and everything will work.
In my case, it's described here: https://docs.hortonworks.com/HDPDocuments/HDP2/HDP-2.3.0/bk_installing_manually_book/content/configure_kerberos_for_storm.html. The nimbus authenticates as storm/<nimbus host>#REALM, and the supervisors and outside clients authenticate as storm/REALM. Everything works.
But here in 2017, we have containers and hostnames are no longer static. So how would I Kerberize a service that runs in Docker Data Center (or Kubernetes, etc)? I have to attach an unknown hostname to the server authentication. I imagine I could create a principal for all possible hostnames and dynamically pick the right one at startup based on where the container lives, but that's kludgy.
Am I misunderstanding how Kerberos works? Is there a solution here that I don't see? I see multiple examples online of people running Storm in Docker, but I can't imagine that nobody's clusters are secure.
I don't know Apache Storm or Docker, but based on previous workings with JBOSS in a cluster in which an inbound client could be connecting to any one of a possible number of different hosts, then you would simply assign a virtual name to the entire pool at the load balancer and kerberize the service according to the virtual name instead of individual host name at the host level. So if you're making service foobar available to clients, you need to create a service principal (SPN) of foobar/virtualhostname#REALM in your Directory to kerberize the service with. You assign that SPN to a user account (not a computer account) to give it the flexibility to work with any Kerberized service which uses that SPN. If you are using Active Directory, you must create a keytab with the SPN inside of it, and place the keytab on each host running the kerberized service instance foobar/virtualhostname#REALM.
I'm trying to get RabbitMQ configured behind an F5 load balancer. I have a working RabbitMQ node with the default node name of rabbit#%computername%. It's set to listen on all network interfaces (all IP addresses 0.0.0.0:5671 which is the AMQP SSL port), and it's working fine. However, all client applications that connect to it are currently using the specific node name e.g. "%computername%". In order to take advantage of the fault tolerance of the load balancer, I want to update all my client applications to use the load-balanced name instead of the specific node name e.g. connect using HostName = "balancedname.mycompany.com" instead of "%computername%". However, when I update my client applications to connect to the load-balanced name, the connection fails. How can I get this to work?
I'm a novice at F5, and I did notice that the pool's members' addresses are IP addresses...should these be the node names instead of the IPs? Is that even possible seeing as the node name can be completely arbitrary and doesn't necessarily map to anything that's network-resolveable? I'm in a hosting situation where I don't have write access to the F5, so trying these things out is a bit tricky.
I haven't found very much information at all on load balancing a RabbitMQ setup. I do understand that all RabbitMQ queues only really exist on one node, and I've set up the F5 in an active-passive mode so that traffic will always route to the primary node unless it goes down.
Update 1: It seems that this issue came back to bite me here. I'm using EXTERNAL authentication using an SSL certificate, and since clients were connecting using the load balance name instead of the node name, and the load balance name was NOT used to create the certificate, it was rejecting the connection. I ended up re-generating the certificate and using the load balance name, but that wasn't enough - I also had to add an entry in the Windows hosts file to map 127.0.0.1 and ::1 to the load balance DNS address.
Update 2: Update 1 solves connection problems only for running client applications on the app server that is part of the load balancer, but remote clients don't work. Inner exception says "The certificate chain was issued by an authority that is not trusted". RabbitMQ + SSL is hard. And adding load balancing makes it even harder.
I'm answering my own question in the hopes that it will save folks some time. In my scenario, I needed for clients to connect to a load balanced address like myrabbithost.mycompany.com, and for the F5 to direct traffic to one node as long as it's up and failover to the secondary node if it's down. I had already configured security and was authenticating to RabbitMQ using self-signed certificates. Those certificates had common names specific to each host which was the problem. In order to work with .NET, the common name on the certificate must match the server name being connected to (myrabbithost.mycompany.com in my case). I had to do the following:
Generate new server and client certificates on the RabbitMQ servers with common names of myrabbithost.mycompany.com
Generate a new certificates for the clients to use while connecting in order to use SSL authentication
Still on the RabbitMQ servers, I had to concatenate the multiple cacert.pem files used for the certificate authority so that clients can authenticate to any node using a client certificate generated by any node. When I modified rabbit.config to use the "all.pem" instead of "cacert.pem", clients were able to connect, but it broke the management UI, so I modified the rabbitmq_management settings in rabbit.config to specific the host-specific cacert.pem file and it started working again.
In order to set up high availability, I set up a RabbitMQ cluster, but ran into some problems there as well. In addition to copying the Erlang cookie from the primary node to the secondary node at C:\Windows and C:\users\myusername, I had to kill the epmd.exe process via task manager as the rabbitmqctl join_cluster command was failing with a "node down" error. The epmd.exe process survives RabbitMQ stoppages and it can cause rabbitmqctl.bat to report erroneous errors like "node down" even when it's not down.
So I've set up my EC2 chef node in several ways (bootstrapping with knife or through client-chef parameters from my node), and every time I try to access the node through knife ssh I get the following error:
WARNING: Failed to connect to *node's FQDN* -- SocketError: getaddrinfo: nodename nor servname provided, or not known
I use knife ssh mainly to update to node and just run sudo client-chef
From this error I assume that I have no access to the FQDN as its an internal address, isn't the chef server supposed to do that for me?
I will soon have a private VPC on AWS so on any occasion I won't be able to access the internal address from my workstation.
Is there a way to make the chef server run this ssh command, or run it any other way?
What I've discovered is basically my misunderstanding of how chef works, I was looking for some sort of push mechanism, and chef does not support push out of the box.
There are 2 workarounds to this:
1) Chef's push jobs - as I'm writing this post, chef push jobs do not work on Ubuntu 14, and I'm not too keen on letting this service dictate the OS of my choice
2) Not recommended anywhere, but installing knife on my chef server worked. Since the chef server is within the VPC, he's my only point of access and from there I'll run knife ssh to all my other nodes.
If anyone is looking for more of a push-based service I'd recommend to look at SaltStack
Since your node does not have an external IP, you should use an ssh gateway. Please refer to this thread: Using knife ec2 plugin to create VM in VPC private subnet
As you mentioned in your answer, the chef doesn't provide push capability, instead it uses a pull. And knife ssh does exactly that - it ssh to the nodes and allows you to run chef-client command which would pull the configuration from the chef server.
Please note, that in your 2nd solution, any node within the VPC with knife would do. This doesn't have to be a Chef server, or should I say the Chef server doesn't have to be in this VPC at all. However, a solution like this compromises security since your authentication with the chef server and your ssh private key would be both located somewhere outside your workstation.
There is also one more way to mention, which is to add chef-client runs to cron if your strategy is well tested.
I have two instances on AWS. One for Cassandra 2.2.5 (it should growth to cluster later) and one for my Java application. Now these nodes connects without SSL, so now I want to enable it.
I tried to follow few instructions but always have exceptions when I'm trying to connect to cluster with SSL from my app and from cqlsh too.
Looking on exception's I suppose that problem in node-names, because instructions are written for case when all computers (cassandra nodes, app-instance and my computer where I user cqlsh) have domain names. But I have only IP's for all nodes. And for Cassandra node I even use two IP: "public" and "AWS private". First I'm using for cqlsh and second for app-to-C* communication.
Of course I have also an URL like "ec2-xx-xx-xx-xx.aws-location-x.compute.amazonaws.com" but I'm not sure it is good idea to use this url.
So, how I should create a keys&certificates to my case?
I set up a NLB cluster given two servers (WS 2008 R2). Each server has one NIC card which I set up for a static ip address. I assigned the cluster an internet name (MyCluster), and assigned it a static ip address. The third box is acting as a client sending TCP data (over WCF) to the cluster's IP I configured (static IP). I am observing the NLB cluster from the NLB manager at one of the nodes - both nodes are green, say started. However, I am only able to see traffic coming in to one of the NLB servers. When I suspend it, I see traffic going to the other NLB server, and so on. I was expecting traffic to be split equally between them. I can't figure out what I missed, any tips please?
If you need more detailed information please ask, not sure how much detail to put in here.
Thanks/.
By default, a port rule created with a Filtering mode of multiple host will use single affinity. In other words, multiple requests from the same client will get directed to the same host. To see traffic going to both hosts try accessing the cluster from multiple clients. You could also set the affinity to "none", but this can lead to other problems.
There's good information on the affinity parameter and how to use it in the NLB help file.