Would like to assign multiple variables from split() on one line of code - variables

Given the following array:
// use strings only in the form <protocol>-<port>
ports: [
'tcp-1514',
'tcp-8080',
'tcp-8443',
],
I'm trying to write jsonnet to split each element of the array to generate this object (represented here in yaml):
ports:
- name: "tcp-1514"
containerPort: 1514
protocol: "tcp"
- name: "tcp-8080"
containerPort: 8080
protocol: "tcp"
- name: "tcp-8443"
containerPort: 8443
protocol: "tcp"
I've tried several iterations of array comprehension to do this, mind you I'm brand new to jsonnet. The latest iteration was something like:
ports: [
{
local proto, port ::= std.split(port_obj, '-');
name: port_obj,
containerPort: port,
protocol: proto,
} for port_obj in $.sharedConfig.ports,
]
where $.sharedConfig.ports is the ports assignment. The problem is local proto, port ::= std.split(port_obj, '-');. I'm not sure this is valid code. The interpreter is poopooing it and I can't find any examples or documentation showing that this is valid.
Ultimately, if it's not valid then I'll have to split() twice, but that would be unfortunate. For instance, this works:
{
local ports = ['tcp-1514', 'tcp-8080', 'tcp-8443',],
ports: [
local port = std.split(name,'-')[1];
local proto = std.split(name,'-')[0];
{
name: name,
protocol: proto,
containerPort: port,
}
for name in ports],
}
which yields:
{
"ports": [
{
"containerPort": "1514",
"name": "tcp-1514",
"protocol": "tcp"
},
{
"containerPort": "8080",
"name": "tcp-8080",
"protocol": "tcp"
},
{
"containerPort": "8443",
"name": "tcp-8443",
"protocol": "tcp"
}
]
}
and YAML:
---
ports:
- containerPort: '1514'
name: tcp-1514
protocol: tcp
- containerPort: '8080'
name: tcp-8080
protocol: tcp
- containerPort: '8443'
name: tcp-8443
protocol: tcp
...but I really dislike the two-line variable assignment. The more I've tested this, the more I believe I'm right determining that the single-line assignment is not doable.
Anyone able to show me how I'm wrong, I'd truly appreciate it.

It may look like a simple answer (that you may have already considered), but well here it goes: using a single local var to hold the split() result, then refer to it in fields' assignments ->
Simple answer:
{
local ports = ['tcp-1514', 'tcp-8080', 'tcp-8443'],
ports: [
local name_split = std.split(name, '-');
{
name: name,
protocol: name_split[0],
containerPort: name_split[1],
}
for name in ports
],
}
Obfuscated answer (no interim local w/split() result):
// Return a map from zipping arr0 (keys) and arr1 (values)
local zipArrays(arr0, arr1) = std.foldl(
// Merge each (per-field) object into a single obj
function(x, y) x + y,
// create per-field object, e.g. { name: <name> },
std.mapWithIndex(function(i, x) { [arr0[i]]: x }, arr1),
{},
);
{
local ports = ['tcp-1514', 'tcp-8080', 'tcp-8443'],
// Carefully ordered set of fields to "match" against: [name] + std.split(...)
local vars = ['name', 'protocol', 'containerPort'],
ports: [
zipArrays(vars, [name] + std.split(name, '-'))
for name in ports
],
}

Related

How can I create router and load balance service added to traefik via consulCatalog?

I have nextcloud running on bare metal 2 nodes:
node1: 192.168.1.10
node2: 192.168.1.11
In the consul I have defined nextcloud service as such on both the nodes:
{
"service": {
"name": "nextcloud",
"tags": ["nextcloud", "traefik"],
"port": 80,
"check": {
"tcp": "localhost:80",
"args": ["ping", "-c1", "127.0.0.1"],
"interval": "10s",
"status": "passing",
"success_before_passing": 3,
"failures_before_critical": 3
}
}
now this shows up in consul fine:
static config: traefik.yaml
global:
# Send anonymous usage data
sendAnonymousUsage: true
api:
dashboard: true
debug: true
log:
level: DEBUG
entryPoints:
http:
address: ":80"
https:
address: ":443"
serversTransport:
insecureSkipVerify: true
providers:
docker:
endpoint: "unix:///var/run/docker.sock"
exposedByDefault: false
file:
directory: "/config/"
watch: true
consulCatalog:
defaultRule: "Host(`{{ .Name }}.sub.mydomain.com`)"
endpoint:
address: http://127.0.0.1:8500
certificatesResolvers:
linode:
acme:
caServer: https://acme-staging-v02.api.letsencrypt.org/directory
email: myemail#domain.com
storage: acme.json
dnsChallenge:
provider: linode
resolvers:
- "1.1.1.1:53"
- "1.0.0.1:53"
and then dynamic /config/config.yaml:
http:
routers:
nextcloud#consulCatalog:
entryPoints:
- "https"
rule: "Host(`home.sub.mydomain.com`) && Path(`/nextcloud`)"
tls:
certResolver: linode
service: nextcloud
services:
nextcloud:
loadBalancer:
servers:
- url: http://192.168.1.10
- url: http://192.168.1.11
passHostHeader: true
but this shows up as file provider with TLS in instead in addtion to exisiting consulcatalog provider.
and not IP or domain mapped.
actual consulcatalog provider showing up but no tls
I am wondering why my dynamic configuration in http did not updated the nextcloud#consulcatalog and set the https entrypoint.
Any help will be greatly appreciated, I am struggling very hard to get this to work.
I have tried following the docs on traefik but its very confusing specially on the consulcatalog part.
Your configuration is showing up as being defined via the file provider because you are statically defining it in the file at /config/config.yaml.
In order to dynamically retrieve this configuration from Consul, you should not be defining the static config file and instead configure tags on the Consul service registrations that will instruct Traefik to route traffic to your service.
For example:
{
"service": {
"name": "nextcloud",
"tags": [
"nextcloud",
"traefik.enable=true",
"traefik.http.routers.nextcloud.entrypoints=https",
"traefik.http.routers.nextcloud.rule=(Host(`home.sub.mydomain.com`) && Path(`/nextcloud`))",
"traefik.http.routers.nextcloud.tls.certresolver=linode",
"traefik.http.services.nextcloud.loadbalancer.passhostheader=true"
],
"port": 80,
"check": {
"tcp": "localhost:80",
"args": [
"ping",
"-c1",
"127.0.0.1"
],
"interval": "10s",
"status": "passing",
"success_before_passing": 3,
"failures_before_critical": 3
}
}
}
More info can be found on the Routing Configuration docs for Traffic's Consul catalog provider.

Accessing values of nested dictionaries

I'm working some separate tasks for automating VM deployments through tower.
Basically I just need a quick run down on how to gather/use the various properties of a registered return from a task.
I've got this.
tasks:
- name: Gather disk info from virtual machine using name
vmware_guest_disk_info:
hostname: "{{ vcenter }}"
username: "{{ username }}"
password: "{{ esxipassword }}"
datacenter: "{{ datacenter }}"
name: "{{ fqdn }}"
register: disk_info
- debug:
var: disk_info
This spits out the information I want. But, for the life of me I can't figure out how to select a single property. can someone tell me how to do that (particularly for the backing_filename) property?
I mean in powershell it would just be disk_info.backing_filename or something like backing = $disk_info | select -expandproperty backing_filename. Just looking for something like the equivalent of that.
Snip of output
{
"disk_info": {
"guest_disk_info": {
"0": {
"key": 2000,
"label": "Hard disk 1",
"summary": "104,857,600 KB",
"backing_filename": "[datastorex] vmname/vmname.vmdk",
To be fair, this one is not as simple as it looks, because your dictionary has a key being a string 0, but, would you be doing disk_info.guest_disk_info.0.backing_filename you would try to access an element 0, so a list, and not a dictionary key '0'.
Here would be an example playbook solving your issue:
- hosts: all
gather_facts: yes
tasks:
- debug:
var: disk_info.guest_disk_info['0'].backing_filename
vars:
disk_info:
guest_disk_info:
'0':
key: 2000
label: Hard disk 1
summary: 104,857,600 KB
backing_filename: "[datastorex] vmname/vmname.vmdk"
That gives:
{
"disk_info.guest_disk_info['0'].backing_filename": "[datastorex] vmname/vmname.vmdk"
}
While this works also, you would see that the YAML is representing a totally different structure, also including a list, and not only multiple nested dictionaries:
- hosts: all
gather_facts: yes
tasks:
- debug:
var: disk_info.guest_disk_info.0.backing_filename
vars:
disk_info:
guest_disk_info:
- key: 2000
label: Hard disk 1
summary: 104,857,600 KB
backing_filename: "[datastorex] vmname/vmname.vmdk"
To give you an equivalent in JSON, since you seems to have issue understanding the YAML constructions, your output is
{
"disk_info": {
"guest_disk_info": {
"0": {
"backing_filename": "[datastorex] vmname/vmname.vmdk"
}
}
}
}
That would be accessible via disk_info.guest_disk_info['0'].backing_filename.
While
{
"disk_info": {
"guest_disk_info": [
{
"backing_filename": "[datastorex] vmname/vmname.vmdk"
}
]
}
}
Would be accessible via disk_info.guest_disk_info.0.backing_filename

Webdriverio Selenium Standalone Service v6 onwards - unable to overwrite the hostname for private Selenium backend

Webdriverio Test Runner has an option
- if you are using a private Selenium backend, you should define the hostname, port, and path here.
hostname: 'localhost',
port: 4444,
path: '/',
Since version: "#wdio/selenium-standalone-service": "^6.0.0"
This "hostname" is unchangeable and always stay as localhost. It seems to autodetect that it should be localhost only and does not refer to config at all i.e. even if I update manually in wdio.conf.js as
hostname: 'selenium-hub',
port: 4445,
path: '/',
Upon execution, still the hostname stays 'localhost' instead of being 'selenium-hub' and port stays as '4444' instead of '4445'
In previous version the command line value with --hostname was getting overwritten successfully as required
i.e. ./node_modules/.bin/wdio wdio.conf.js --hostname 'selenium-hub'
would pass selenium-hub as hostname successfully....
anyone experiencing similar issue ?
add the hostname, port, and path to the capabilities array.
instead of:
hostname: '{ unique ip address}',
port: { port number },
path: {'/'},
protocol: '{http' || 'https'},
capabilities: [{
maxInstances: 5,
browserName: 'chrome',
}],
Do this:
capabilities: [{
maxInstances: 5,
browserName: 'chrome',
hostname: '{ unique ip address}',
port: { port number },
path: {'/'},
protocol: '{http' || 'https'},
}],

Service discovery with eureka is not working in docker container

When I run my API gateway in docker container then it is not able to find my services which are registered in eureka.
API Gateway
-- ocelot.json
{
"ReRoutes": [
{
"DownstreamPathTemplate": "/api/values",
"DownstreamScheme": "http",
"UseServiceDiscovery": true,
"ServiceName": "sampleservice",
"UpstreamPathTemplate": "/sample-api/{catchAll}"
}
],
"GlobalConfiguration": {
"UseServiceDiscovery": true,
"ServiceDiscoveryProvider": {
"Type": "Eureka",
"Host": "myeurekaserver",
"Port": "8761"
}
}
}
-- appsettings.json for API Gateway
{
"eureka": {
"client": {
"shouldRegisterWithEureka": false,
"serviceUrl": "http://myeurekaserver:8761/eureka/",
"ValidateCertificates": false
},
"instance": {
"appName": "gateway",
"hostName": "myeurekaserver",
"port": "7000"
}
}
}
Service Configuration --appsettings.json
{
"eureka": {
"client": {
"shouldRegisterWithEureka": true,
"serviceUrl": "http://myeurekaserver:8761/eureka/",
"ValidateCertificates": false
},
"instance": {
"appName": "sampleservice",
"hostName": "myeurekaserver",
"port": "7001"
}
}
}
docker-compose.yml
version: '3.4'
services:
sampleapi:
image: ${DOCKER_REGISTRY-}sampleapi
ports:
- "7001:80"
networks:
- ecnetwork
build:
context: .
dockerfile: SampleAPI/Dockerfile
gateway:
image: ${DOCKER_REGISTRY-}gateway
ports:
- "7000:80"
networks:
- ecnetwork
build:
context: .
dockerfile: Gateway/Dockerfile
myeurekaserver:
image: ${DOCKER_REGISTRY-}myeurekaserver
ports:
- "8761:8761"
networks:
- ecnetwork
build:
context: .
dockerfile: MyEurekaServer/Dockerfile
networks:
ecnetwork:
external: true
When I run command docker-compose up and check on http://localhost:8761/ I find my services have been registred in the eureka server, but I run http://localhost:7000/sample-api/order
It returns
localhost is currently unable to handle this request. HTTP ERROR 500
I checked my console window, then It is API gateway is able to discover the services, here is the log.
gateway_1 | dbug: Steeltoe.Discovery.Eureka.DiscoveryClient[0]
gateway_1 | FetchRegistryDelta returned: OK
gateway_1 | dbug: Steeltoe.Discovery.Eureka.DiscoveryClient[0]
gateway_1 | FetchRegistry succeeded
It's an application error, check your app API gateway.
500 Internal Server Error
A generic error message, given when an unexpected condition was encountered and no more specific message is suitable
Try to debug your application without Docker.
Check in the docker on which port the service is registered 7000 or 80?
Then see if the 7000 port is accessible for you in local by telnet

How to parameterize ports in OpenShift JSON Project Template

I'm trying to create a custom project template in OpenShift Origin. The Service configuration specifically, looks like below:
{
"kind": "Service",
"apiVersion": "v1",
"metadata": {
"name": "${NAME}",
"annotations": {
"description": "Exposes and load balances the node.js application pods"
}
},
"spec": {
"ports": [
{
"name": "web",
"port": "${APPLICATION_PORT}",
"targetPort": "${APPLICATION_PORT}",
"protocol": "TCP"
}
],
"selector": {
"name": "${NAME}"
}
}
},
where, APPLICATION_PORT is supplied as a user parameter:
"parameters": [
{
"name": "APPLICATION_PORT",
"displayName": "Application Port",
"description": "The exposed port that will route to the node.js application",
"value": "8000"
},
When I try to use this template to create a project, I get the following error:
spec.ports[0].targetPort: Invalid value: "8000": must be an IANA_SVC_NAME (at most 15 characters, matching regex [a-z0-9]([a-z0-9-]*[a-z0-9])*...
I get a similar error in my DeploymentConfig as well, for the http ports in the liveness and readiness probes:
"readinessProbe": {
"timeoutSeconds": 3,
"initialDelaySeconds": 3,
"httpGet": {
"path": "/Info",
"port": "${APPLICATION_ADMIN_PORT}"
}
},
"livenessProbe": {
"timeoutSeconds": 3,
"initialDelaySeconds": 30,
"httpGet": {
"path": "/Info",
"port": "${APPLICATION_ADMIN_PORT}"
}
},
where, APPLICATION_ADMIN_PORT, again, is user-supplied.
Error:
spec.template.spec.containers[0].livenessProbe.httpGet.port: Invalid value: "8001": must be an IANA_SVC_NAME...
spec.template.spec.containers[0].readinessProbe.httpGet.port: Invalid value: "8001": must be an IANA_SVC_NAME...
I've been following https://blog.openshift.com/part-2-creating-a-template-a-technical-walkthrough/ to understand templates, and it, unfortunately, does not have any examples of ports being parameterized anywhere.
It almost seems as if strings are not allowed as the values of these ports. Is that the case? What's the right way to parameterize these values? Should I switch to YAML?
Versions:
OpenShift Master: v1.1.6-3-g9c5694f
Kubernetes Master: v1.2.0-36-g4a3f9c5
Edit 1: I tried the same configuration in YAML format, and got the same error. So, JSON vs YAML is not the issue.
Unfortunately it is not currently possible to parameterize non-string field values: https://docs.openshift.org/latest/dev_guide/templates.html#writing-parameters
" Parameters can be referenced by placing values in the form "${PARAMETER_NAME}" in place of any string field in the template."
Templates are in the process of being upstreamed to Kubernetes and this limitation is being addressed there:
https://github.com/kubernetes/kubernetes/blob/master/docs/proposals/templates.md
The proposal is being implemented in PRs 25622 and 25293 in the kubernetes repo.
edit:
Templates now support non-string parameters as documented here: https://docs.openshift.org/latest/dev_guide/templates.html#writing-parameters
I don't know if this option was available in 2016 when this post was added but now you can use ${{PARAMETER_NAME}} to parameterize non-string field values.
spec:
externalTrafficPolicy: Cluster
ports:
- name: ${NAME}-port
port: ${{PORT_PARAMETER}}
protocol: TCP
targetPort: ${{PORT_PARAMETER}}
sessionAffinity: None
This may a be a bad practice but I'm using sed to substitute int parameters:
cat template.yaml | sed -e 's/PORT/8080/g' > proxy-template-subst.yaml
Template:
apiVersion: template.openshift.io/v1
kind: Template
objects:
- apiVersion: v1
kind: Service
metadata:
name: ${NAME}
namespace: ${NAMESPACE}
spec:
externalTrafficPolicy: Cluster
ports:
- name: ${NAME}-port
port: PORT
protocol: TCP
targetPort: PORT
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
parameters:
- description: Desired service name
name: NAME
required: true
value: need_real_value_here
- description: IP adress
name: IP
required: true
value: need_real_value_here
- description: namespace where to deploy
name: NAMESPACE
required: true
value: need_real_value_here