Custom labels/constants in Alertmanager config - variables

I have an Alertmanager config file which defines many alerts. All of these alerts use the same time range in their Prometheus expressions and messaging. Is there way I can define a constant once in the file, which is then used throughout. That way, if I ever change the value of this constant, it's only required to be changed in one place?
Here's an illustration of what I am trying to achieve:
apiVersion: monitoring.coreos.com/v1
kind: PrometheusRule
metadata:
name: kube-prometheus-stack-services
namespace: monitoring
labels:
app: kube-prometheus-stack
release: kube-prometheus-stack
customer_variables:
time_range: 5m # CUSTOM VARIABLE DEFINED HERE
spec:
groups:
- name: Service1
rules:
- alert: Service1Alert1
annotations:
summary: "Some summary of alert1"
message: "Some error rate has increased to {{ $value }}% in the last {{ $customer_variables.time_range }}." # CUSTOM VARIABLE USED HERE
expr: # some expression using my_metric_1{label="..."}[{{ $customer_variables.time_range }}] # CUSTOM VARIABLE USED HERE
for: {{ $customer_variables.time_range }} # CUSTOM VARIABLE USED HERE
labels:
severity: critical
- alert: Service1Alert2
annotations:
summary: "Some summary of alert2"
message: "Some other error rate has increased to {{ $value }}% in the last {{ $customer_variables.time_range }}." # CUSTOM VARIABLE USED HERE
expr: # some expression using my_metric_1{label="..."}[{{ $customer_variables.time_range }}] # CUSTOM VARIABLE USED HERE
for: {{ $customer_variables.time_range }} # CUSTOM VARIABLE USED HERE
labels:
severity: critical

Related

GitHub Actions: Use variables in matrix definition?

I have the following code in a GitHub Action config:
name: Build & Tests
on:
pull_request:
env:
CARGO_TERM_COLOR: always
ZEROCOPY_MSRV: 1.61.0
ZEROCOPY_CURRENT_STABLE: 1.64.0
ZEROCOPY_CURRENT_NIGHTLY: nightly-2022-09-26
jobs:
build_test:
runs-on: ubuntu-latest
strategy:
matrix:
# See `INTERNAL.md` for an explanation of these pinned toolchain
# versions.
channel: [ ${{ env.ZEROCOPY_MSRV }}, ${{ env.ZEROCOPY_CURRENT_STABLE }}, ${{ env.ZEROCOPY_CURRENT_NIGHTLY }} ]
target: [ "i686-unknown-linux-gnu", "x86_64-unknown-linux-gnu", "arm-unknown-linux-gnueabi", "aarch64-unknown-linux-gnu", "powerpc-unknown-linux-gnu", "powerpc64-unknown-linux-gnu", "wasm32-wasi" ]
features: [ "" , "alloc,simd", "alloc,simd,simd-nightly" ]
exclude:
# Exclude any combination which uses a non-nightly toolchain but
# enables nightly features.
- channel: ${{ env.ZEROCOPY_MSRV }}
features: "alloc,simd,simd-nightly"
- channel: ${{ env.ZEROCOPY_CURRENT_STABLE }}
features: "alloc,simd,simd-nightly"
I'm getting the following parsing error on this file:
Invalid workflow file: .github/workflows/ci.yml#L19
You have an error in your yaml syntax on line 19
It appears to be referring to this line (it's actually one off, but maybe it's zero-indexing its line numbers?):
channel: [ ${{ env.ZEROCOPY_MSRV }}, ${{ env.ZEROCOPY_CURRENT_STABLE }}, ${{ env.ZEROCOPY_CURRENT_NIGHTLY }} ]
Is there any way to use variables in the matrix definition like this? Or do I just need to hard-code everything?
According to the documentation (reference 1 and reference 2)
Environment variables (at the workflow level) are available to the steps of all jobs in the workflow.
In your example, the environment variables are used at the job level (inside the job strategy / matrix definition), not inside the job steps.
At that level, environment variables aren't interpolated by the GitHub interpreter.
First alternative
Hardcode the values inside the channel field inside your matrix strategy:
Example:
channel: [ "1.61.0", "1.64.0", "nightly-2022-09-26" ]
However, you'll have to do this for each job (bad for maintenance as duplicated code).
Second alternative
Use inputs (with reusable workflow workflow_call trigger, or with workflow_dispatch trigger.
Example:
on:
workflow_dispatch: # or workflow_call
inputs:
test1:
description: "Test1"
required: false
default: "test1"
test2:
description: "Test2"
required: false
default: "test2"
test3:
description: "Test3"
required: false
default: "test3"
jobs:
build_test:
runs-on: ubuntu-latest
strategy:
matrix:
channel: [ "${{ inputs.test1 }}", "${{ inputs.test2 }}", "${{ inputs.test3 }}" ]
In that case, inputs will be interpolated by the GitHub interpreter.
However, you'll need to trigger the workflow from another workflow, or through the GitHub API to send the inputs (in some way, it gives you more flexibility with the values, but increase the complexity).

Tekton trigger flow from github

I am learning Tekton (for business), coming from github actions (private).
The Tekton docs (or any other tutorial I could find) have instructions on how to automatically start a pipeline from a github push. Basically they all somewhat follow the below flow: (I am aware of PipelineRun/TaskRun etc)
Eventlistener - Trigger - TriggerTemplate - Pipeline
All above steps are basically configuration steps you need to take (and files to create and maintain), one easier than the other but as far as I can see they also need to be taken for every single repo you're maintaining. Compared to github actions where I just need 1 file in my repo describing everything I need this seems very elaborate (if not cumbersome).
Am I missing something ? Or is this just the way to go ?
Thanks !
they also need to be taken for every single repo you're maintaining
You're mistaken here.
The EventListener receives the payload of your webhook.
Based on your TriggerBinding, you may map fields from that GitHub payload, to variables, such as your input repository name/URL, a branch or ref to work with, ...
For GitHub push events, one way to do it would be with a TriggerBinding such as the following:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
name: github-push
spec:
params:
- name: gitbranch
value: $(extensions.branch_name) # uses CEL interceptor, see EL below
- name: gitrevision
value: $(body.after) # uses body from webhook payload
- name: gitrepositoryname
value: $(body.repository.name)
- name: gitrepositoryurl
value: $(body.repository.clone_url)
We may re-use those params within our TriggerTemplate, passing them to our Pipelines / Tasks:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: github-pipelinerun
spec:
params:
- name: gitbranch
- name: gitrevision
- name: gitrepositoryname
- name: gitrepositoryurl
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: github-job-
spec:
params:
- name: identifier
value: "demo-$(tt.params.gitrevision)"
pipelineRef:
name: ci-docker-build
resources:
- name: app-git
resourceSpec:
type: git
params:
- name: revision
value: $(tt.params.gitrevision)
- name: url
value: $(tt.params.gitrepositoryurl)
- name: ci-image
resourceSpec:
type: image
params:
- name: url
value: registry.registry.svc.cluster.local:5000/ci/$(tt.params.gitrepositoryname):$(tt.params.gitrevision)
- name: target-image
resourceSpec:
type: image
params:
- name: url
value: registry.registry.svc.cluster.local:5000/ci/$(tt.params.gitrepositoryname):$(tt.params.gitbranch)
timeout: 2h0m0s
Using the following EventListener:
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
name: github-listener
spec:
triggers:
- name: github-push-listener
interceptors:
- name: GitHub push payload check
github:
secretRef:
secretName: github-secret # a Secret you would create (option)
secretKey: secretToken # the secretToken in my Secret matches to secret configured in GitHub, for my webhook
eventTypes:
- push
- name: CEL extracts branch name
ref:
name: cel
params:
- name: overlays
value:
- key: truncated_sha
expression: "body.after.truncate(7)"
- key: branch_name
expression: "body.ref.split('/')[2]"
bindings:
- ref: github-push
template:
ref: github-pipelinerun
And now, you can expose that EventListener, with an Ingress, to receive notifications from any of your GitHub repository.

sslCertificateCouldNotParseCert - Error syncing to GCP:

I have a terraform code which will deploy the frontend application and have ingress.yaml helm chart.
ingress.yaml
{{- if .Values.ingress.enabled -}}
{{- if semverCompare ">=1.14-0" .Capabilities.KubeVersion.GitVersion -}}
apiVersion: networking.k8s.io/v1beta1
{{- else -}}
apiVersion: extensions/v1beta1
{{- end }}
kind: Ingress
metadata:
name: {{ .Values.global.namespace }}-ingress
namespace: {{ .Values.global.namespace }}
labels:
{{- include "test-frontend.labels" . | nindent 4 }}
annotations:
kubernetes.io/ingress.class: "gce-internal"
kubernetes.io/ingress.allow-http: "false"
spec:
{{- if .Values.ingress.tls }}
tls:
{{- range .Values.ingress.tls }}
- hosts:
{{- range .hosts }}
- {{ . | quote }}
{{- end }}
secretName: {{ .secretName }}
{{- end }}
{{- end }}
rules:
{{- range .Values.ingress.hosts }}
- host: {{ .host | quote }}
http:
paths:
{{- range .paths }}
- path: {{ .path }}
backend:
serviceName: {{ .servicename }}
servicePort: {{ .serviceport }}
{{- end }}
{{- end }}
{{- end }}
values.yaml
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: "gce-internal"
kubernetes.io/ingress.regional-static-ip-name : "ingress-internal-static-ip"
kubernetes.io/ingress.allow-http: "false"
hosts:
- host: test-dev.test.com
paths:
- path: "/*"
servicename: test-frontend-service
serviceport: 80
- path: "/api/*"
servicename: test-backend-service
serviceport: 80
tls:
- hosts:
- test-dev.test.com
secretName: ingress-tls-credential-file
type: kubernetes.io/tls
crt: <<test.pem value>>
key: <<test.key value>>
The terraform apply command ran successfully. In GCP also the certificate is accepted and ingress in up and running inside Kubernetes Service in GCP. But If I pass the .crt and .key as a file in values.yaml in terraform code
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: "gce-internal"
kubernetes.io/ingress.regional-static-ip-name : "ingress-internal-static-ip"
kubernetes.io/ingress.allow-http: "false"
hosts:
- host: test-dev.test.com
paths:
- path: "/*"
servicename: test-frontend-service
serviceport: 80
- path: "/api/*"
servicename: test-backend-service
serviceport: 80
tls:
- hosts:
- test-dev.test.com
secretName: ingress-tls-credential-file
type: kubernetes.io/tls
crt: file(../../.secret/test.crt)
key: file(../../.secret/test.key)
The values.yaml will send the certificate to helm->template->secret.yaml which will create the secrets(ingress-tls-credential-file)
secret.yaml
{{- if .Values.ingress.tls }}
{{- $namespace := .Values.global.namespace }}
{{- range .Values.ingress.tls }}
apiVersion: v1
kind: Secret
metadata:
name: {{ .secretName }}
namespace: {{ $namespace }}
labels:
{{- include "test-frontend.labels" $ | nindent 4 }}
type: {{ .type }}
data:
tls.crt: {{ toJson .crt | b64enc | quote }}
tls.key: {{ toJson .key | b64enc | quote }}
{{- end }}
{{- end }}
We are getting below error in GCP -> Kubernetes Engine -> Service & Ingress. How to pass the files to the values.yaml file.
Error syncing to GCP: error running load balancer syncing routine:
loadbalancer 6370cwdc-isp-isp-ingress-ixjheqwi does not exist: Cert
creation failures - k8s2-cr-6370cwdc-q0ndkz9m629eictm-ca5d0f56ba7fe415
Error:googleapi: Error 400: The SSL certificate could not be parsed.,
sslCertificateCouldNotParseCert
So google can accept your cert and key files, you need to make sure they have the proper format as per next steps
You need to first format them creating a Self Managed SSL Certificate resource with your existing files using you GCP Cloud Shell
gcloud compute ssl-certificates create CERTIFICATE_NAME \
--certificate=CERTIFICATE_FILE \
--private-key=PRIVATE_KEY_FILE \
--region=REGION \
--project=PROJECT_ID
Then you need to complete a few more steps to make sure you have all the parameters required in your .yaml file and that you have the proper services enable to accept the information coming from it (you may already have completed them):
Enable Kubernetes Engine API by running the following command:
gcloud services enable container.googleapis.com \
--project=PROJECT_ID
Create a GKE cluster:
gcloud container clusters create CLUSTER_NAME \
--release-channel=rapid \
--enable-ip-alias \
--network=NETWORK_NAME \
--subnetwork=BACKEND_SUBNET_NAME \
--scopes=https://www.googleapis.com/auth/cloud-platform \
--region=REGION --machine-type=MACHINE_TYPE \
--project=PROJECT_ID
The cluster is created in the BACKEND_SUBNET_NAME.
The cluster uses GKE version is 1.18.9-gke.801 or later.
The cluster is created with the Cloud Platform scope.
The cluster is created with the desired service account you would like to use to run the - application.
The cluster is using n1-standard-4 machine type or better.
Enable IAP by doing the following steps:
Configure the OAuth consent screen.
Create OAuth credentials.
Convert the ID and Secret to base64 by running the following commands:
echo -n 'CLIENT_ID' | base64
echo -n 'CLIENT_SECRET' | base64
Create an internal static IP address, and reserve a static IP address for your load balancer
gcloud compute addresses create STATIC_ADDRESS_NAME \
--region=REGION --subnet=BACKEND_SUBNET_NAME \
--project=PROJECT_ID
Get the static IP address by running the following command:
gcloud compute addresses describe STATIC_ADDRESS_NAME \
--region=REGION \
--project=PROJECT_ID
7.Create the values YAML file by copying the gke_internal_ip_config_example.yaml and renaming it to PROJECT_ID_gke_config.yaml:
clientIDEncoded: Base64 encoded CLIENT_ID from earlier step.
clientSecretEncoded: Base64 encoded CLIENT_SECRET from earlier step.
certificate.name: CERTIFICATE_NAME that you have created earlier.
initialEmail: The INITIAL_USER_EMAIL email of the initial user who will set up Custom Governance.
staticIpName: STATIC_ADDRESS_NAME that you created earlier.
Try again your deployment after completing above steps.
You seem to mix a secret and a direct definition.
You need first to create the ingress-tls-credential-file secret then link it in your ingress definition like the example https://kubernetes.io/fr/docs/concepts/services-networking/ingress/#tls
apiVersion: v1
data:
tls.crt: file(../../.secret/test.crt)
tls.key: file(../../.secret/test.key)
kind: Secret
metadata:
name: ingress-tls-credential-file
namespace: default
type: kubernetes.io/tls
Then clean your ingress
ingress:
enabled: true
annotations:
kubernetes.io/ingress.class: "gce-internal"
kubernetes.io/ingress.regional-static-ip-name : "ingress-internal-static-ip"
kubernetes.io/ingress.allow-http: "false"
hosts:
- host: test-dev.test.com
paths:
- path: "/*"
servicename: test-frontend-service
serviceport: 80
- path: "/api/*"
servicename: test-backend-service
serviceport: 80
tls:
- hosts:
- test-dev.test.com
secretName: ingress-tls-credential-file
type: kubernetes.io/tls

Conftest Fails For a Valid Kubernetets YAML File

I have the following simple Kubernetes YAML Deployment:
apiVersion: apps/v1
kind: Deployment
metadata:
name: {{ .Values.app.name }}
namespace: {{ .Values.app.namespace }}
spec:
selector:
matchLabels:
app: {{ .Values.app.name }}
replicas: 1
template:
metadata:
labels:
app: {{ .Values.app.name }}
spec:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
containers:
- name: {{ .Values.app.name }}
image: {{ .Values.plantSimulatorService.image.repository }}:{{ .Values.plantSimulatorService.image.tag }}
ports:
- containerPort: {{ .Values.plantSimulatorService.ports.containerPort }} # Get this value from ConfigMap
I have the following in my test.rego:
package main
import data.kubernetes
name = input.metadata.name
deny[msg] {
kubernetes.is_deployment
not input.spec.template.spec.securityContext.runAsNonRoot
msg = sprintf("Containers must not run as root in Deployment %s", [name])
}
When I ran this using the following command:
joesan#joesan-InfinityBook-S-14-v5:~/Projects/Private/infrastructure-projects/plant-simulator-deployment$ helm conftest helm-k8s -p test
WARN - Found service plant-simulator-service but services are not allowed
WARN - Found service plant-simulator-grafana but services are not allowed
WARN - Found service plant-simulator-prometheus but services are not allowed
FAIL - Containers must not run as root in Deployment plant-simulator
FAIL - Deployment plant-simulator must provide app/release labels for pod selectors
As you can see I'm indeed not running the container as root, but despite that I get this error message - Containers must not run as root in Deployment plant-simulator
Any ideas what the reason could be?
You need to add runAsNonRoot to your securityContext:
securityContext:
runAsUser: 1000
runAsGroup: 3000
fsGroup: 2000
runAsNonRoot: true
The rego rule is only able to validate Yaml structure - it is not clever enough to work out that your configuration is effectively running a non-root user.

How to configure the logger of schema registry image with helm chart deployment

I use schema registry 5.4 - https://hub.docker.com/r/confluentinc/cp-schema-registry/, and use helm chart https://github.com/helm/charts/tree/master/incubator/schema-registry to deploy.
To set my logger, I create a configmap and mount it to the schema registry container that uses the image -confluentinc/cp-schema-registry:5.4.0
apiVersion: v1
kind: ConfigMap
metadata:
name: {{ template "schema-registry.fullname" . }}-log4j-configmap
labels:
app: {{ template "schema-registry.name" . }}
chart: {{ template "schema-registry.chart" . }}
release: {{ .Release.Name }}
heritage: {{ .Release.Service }}
data:
log4j.properties: |+
log4j.appender.stdout=org.apache.log4j.ConsoleAppender
log4j.appender.stdout.encoding=UTF-8
But I don't know what should be changed in the command of the container, or there is a variable pointing to log4j.properties to set it?
for example I plan to change the command like the following:
command:
- java
- -Dlog4j.configuration=file:/etc/schema-registry-log4j/log4j.properties
- -jar
- schema-registry.jar
Option 1:
One way I found is to build a new image using FROM and put my custom log4j properties there.
https://docs.confluent.io/current/installation/docker/development.html#log-to-external-volumes
Option 2:
The other way is to set an environment variable
SCHEMA_REGISTRY_LOG4J_LOGGERS="org.apache.kafka=ERROR,io.confluent.rest.exceptions=FATAL"