Tekton build Docker image with Kaniko - please provide a valid path to a Dockerfile within the build context with --dockerfile - tekton

I am new to Tekton (https://tekton.dev/) and I am trying to
Clone the repository
Build a docker image with the Dockerfile
I have a Tekton pipeline and when I try to execute it, I get the following error:
Error: error resolving dockerfile path: please provide a valid path to a Dockerfile within the build context with --dockerfile
Please find the Tekton manifests below:
1. Pipeline.yml
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: clone-read
spec:
description: |
This pipeline clones a git repo, then echoes the README file to the stout.
params:
- name: repo-url
type: string
description: The git repo URL to clone from.
- name: image-name
type: string
description: for Kaniko
- name: image-path
type: string
description: path of Dockerfile for Kaniko
workspaces:
- name: shared-data
description: |
This workspace contains the cloned repo files, so they can be read by the
next task.
tasks:
- name: fetch-source
taskRef:
name: git-clone
workspaces:
- name: output
workspace: shared-data
params:
- name: url
value: $(params.repo-url)
- name: show-readme
runAfter: ["fetch-source"]
taskRef:
name: show-readme
workspaces:
- name: source
workspace: shared-data
- name: build-push
runAfter: ["show-readme"]
taskRef:
name: kaniko
workspaces:
- name: source
workspace: shared-data
params:
- name: IMAGE
value: $(params.image-name)
- name: CONTEXT
value: $(params.image-path)
1. PipelineRun.yml
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: clone-read-run
spec:
pipelineRef:
name: clone-read
podTemplate:
securityContext:
fsGroup: 65532
workspaces:
- name: shared-data
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
# - name: git-credentials
# secret:
# secretName: git-credentials
params:
- name: repo-url
value: https://github.com/iamdempa/tekton-demos.git
- name: image-name
value: "python-test"
- name: image-path
value: $(workspaces.shared-data.path)/BuildDockerImage2
And here's my repository structure:
. . .
.
├── BuildDockerImage2
│ ├── 1.show-readme.yml
│ ├── 2. Pipeline.yml
│ ├── 3. PipelineRun.yml
│ └── Dockerfile
├── README.md
. . .
7 directories, 25 files
Could someone help me what is wrong here?
Thank you

I was able to find the issue. Issue was with the way I have provided the path.
In the kaniko task, the CONTEXT variable determines the path of the Dockerfile. And the default value is set to ./ and with some additional prefix as below:
$(workspaces.source.path)/$(params.CONTEXT)
That mean, the path of the workspaces is already being appended and I don't need to append that part as I mentioned in the image-path value below:
$(workspaces.shared-data.path)/BuildDockerImage2
Instead, I had to put just the folder name as below:
- name: image-path
value: BuildDockerImage2
This fixed the problem I had.

Related

task hello-world has failed: declared workspace "output" is required but has not been bound

apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: hello-world
spec:
workspaces:
- name: output
description: folder where output goes
steps:
- name: hello-world1
image: ubuntu
command: ["/bin/bash"]
args: ["-c", "echo Hello World 1! > $(workspaces.output.path)<200b>/message1.txt"]
- name: hello-world2
image: ubuntu
script: |
#!/usr/bin/env bash
set -xe
echo Hello World 2! > $(workspaces.output.path)/message2.txt
From your error message, we can guess that the TaskRun (and PipelineRun) trying to run this task does not define a workspace to be used with your Task.
Say I would like to call your Task: I would write a Pipeline, which should include something like:
apiVersion: tekton.dev/v1beta1
kind: Pipeline
metadata:
name: hello-world
spec:
tasks:
- name: hello-world-task
taskRef:
name: hello-world
workspaces:
- name: output
workspace: my-workspace
workspaces:
- name: my-workspace
optional: true
And then, start this pipeline with the following PipelineRun:
apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
name: hello-world-0
spec:
pipelineRef: hello-world
workspaces:
- name: my-workspace
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 1Gi
See Tekton Pipelines Workspaces docs.

How to attach a volume to docker running in tekton pipelines

I have a problem attaching a volume to the docker image running inside tekton pipelines. I have used the below task
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: distributor-base
namespace: cicd
labels:
app.kubernetes.io/version: "0.1"
annotations:
tekton.dev/pipelines.minVersion: "0.12.1"
tekton.dev/platforms: "linux/amd64"
spec:
params:
- name: builder_image
description: The location of the docker builder image.
default: docker:stable
- name: dind_image
description: The location of the docker-in-docker image.
default: docker:dind
- name: context
description: Path to the directory to use as context.
default: .
workspaces:
- name: source
steps:
- name: docker-build
image: docker
env:
# Connect to the sidecar over TCP, with TLS.
- name: DOCKER_HOST
value: tcp://localhost:2376
# Verify TLS.
- name: DOCKER_TLS_VERIFY
value: '1'
# Use the certs generated by the sidecar daemon.
- name: DOCKER_CERT_PATH
value: /certs/client
- name: DOCKER_USER
valueFrom:
secretKeyRef:
key: username
name: docker-auth
- name: DOCKER_TOKEN
valueFrom:
secretKeyRef:
key: password
name: docker-auth
- name: DIND_CONFIG
valueFrom:
configMapKeyRef:
key: file
name: dind-env
workingDir: $(workspaces.source.path)
args:
- --storage-driver=vfs
- --debug
securityContext:
privileged: true
script: |
#!/usr/bin/env sh
set -e
pwd
ls -ltr /workspace/source
docker run --privileged -v "/workspace/source:/workspace" busybox ls -ltr /workspace
volumeMounts:
- mountPath: /certs/client
name: dind-certs
sidecars:
- image: $(params.dind_image)
name: server
args:
- --storage-driver=vfs
- --debug
- --userland-proxy=false
resources:
requests:
memory: "512Mi"
securityContext:
privileged: true
env:
# Write generated certs to the path shared with the client.
- name: DOCKER_TLS_CERTDIR
value: /certs
volumeMounts:
- mountPath: /certs/client
name: dind-certs
# Wait for the dind daemon to generate the certs it will share with the
# client.
readinessProbe:
periodSeconds: 1
exec:
command: ['ls', '/certs/client/ca.pem']
volumes:
- name: dind-certs
emptyDir: {}
in the above task workspace comes from another git-clone task
workspaces:
- name: source
in this task, I am trying to run a docker image that has access to the workspace folder , because I have to modify some files in the workspace folder.
when we look into the script
pwd
ls -ltr /workspace/source
docker run --privileged -v "/workspace/source:/workspace"
below is the console log of above 3 commands
workspace/source
total 84
-rwxr-xr-x 1 50381 50381 3206 Jun 1 10:13 README.md
-rwxr-xr-x 1 50381 50381 10751 Jun 1 10:13 Jenkinsfile.next
-rwxr-xr-x 1 50381 50381 5302 Jun 1 10:13 wait-for-it.sh
drwxr-xr-x 4 50381 50381 6144 Jun 1 10:13 overlays
-rwxr-xr-x 1 50381 50381 2750 Jun 1 10:13 example-distributor.yaml
drwxr-xr-x 5 50381 50381 6144 Jun 1 10:13 bases
-rw-r--r-- 1 50381 50381 0 Jun 1 10:13 semantic.out
-rw-r--r-- 1 50381 50381 44672 Jun 1 10:13 final.yaml
Unable to find image 'busybox:latest' locally
latest: Pulling from library/busybox
462eb288b104: Pulling fs layer
462eb288b104: Verifying Checksum
462eb288b104: Download complete
462eb288b104: Pull complete
Digest: sha256:ebadf81a7f2146e95f8c850ad7af8cf9755d31cdba380a8ffd5930fba5996095
Status: Downloaded newer image for busybox:latest
total 0
basically pwd command is giving me results
and ls -ltr command also gives me the results
but when I try to attach /workspace/source folder as a volume to busybox docker, I am not able to see the content.
i mean since I have attached a volume into the directory /workspace I would expect the contents from local folder /workspace/source but I see 0 results from the above log.
basically volume is not getting attached properly.
can anyone please help me to fix this issue.
below is my pipeline run triggered by tekton-triggers
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: github-gitops-template
namespace: cicd
spec:
params:
- name: gitRevision
description: The git revision (SHA)
default: master
- name: gitRepoUrl
description: The git repository url ("https://github.com/foo/bar.git")
- name: gitRepoName
description: The git repository name
- name: branchUrl
description: The git repository branch url
- name: repoFullName
description: The git repository full name
- name: commitSha
description: The git commit sha
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: $(tt.params.gitRepoName)-
spec:
timeout: 0h10m
pipelineRef:
name: gitops-pipeline
serviceAccountName: github-service-account
params:
- name: url
value: $(tt.params.gitRepoUrl)
- name: branch
value: $(tt.params.gitRevision)
- name: repoName
value: $(tt.params.gitRepoName)
- name: branchUrl
value: $(tt.params.branchUrl)
- name: repoFullName
value: $(tt.params.repoFullName)
- name: commitSha
value: $(tt.params.commitSha)
workspaces:
- name: ws
volumeClaimTemplate:
spec:
accessModes:
- ReadWriteOnce
resources:
requests:
storage: 50Mi
below is my task run:
completionTime: '2022-06-01T10:13:47Z'
conditions:
- lastTransitionTime: '2022-06-01T10:13:47Z'
message: All Steps have completed executing
reason: Succeeded
status: 'True'
type: Succeeded
podName: gitops-core-business-tzb7f-distributor-base-pod
sidecars:
- container: sidecar-server
imageID: 'docker-pullable://gcr.io/tekton-releases/github.com/tektoncd/pipeline/cmd/nop#sha256:1d65a20cd5fbc79dc10e48ce9d2f7251736dac13b302b49a1c9a8717c5f2b5c5'
name: server
terminated:
containerID: 'docker://d5e96143812bb4912c6297f7706f141b9036c6ee77efbffe2bcb7edb656755a5'
exitCode: 0
finishedAt: '2022-06-01T10:13:49Z'
message: Sidecar container successfully stopped by nop image
reason: Completed
startedAt: '2022-06-01T10:13:37Z'
startTime: '2022-06-01T10:13:30Z'
steps:
- container: step-docker-build
imageID: 'docker-pullable://docker#sha256:5bc07a93c9b28e57a58d57fbcf437d1551ff80ae33b4274fb60a1ade2d6c9da4'
name: docker-build
terminated:
containerID: 'docker://18aa9111f180f9cfc6b9d86d5ef1da9f8dbe83375bb282bba2776b5bbbcaabfb'
exitCode: 0
finishedAt: '2022-06-01T10:13:46Z'
reason: Completed
startedAt: '2022-06-01T10:13:42Z'
taskSpec:
params:
- default: 'docker:stable'
description: The location of the docker builder image.
name: builder_image
type: string
- default: 'docker:dind'
description: The location of the docker-in-docker image.
name: dind_image
type: string
- default: .
description: Path to the directory to use as context.
name: context
type: string
sidecars:
- args:
- '--storage-driver=vfs'
- '--debug'
- '--userland-proxy=false'
env:
- name: DOCKER_TLS_CERTDIR
value: /certs
image: $(params.dind_image)
name: server
readinessProbe:
exec:
command:
- ls
- /certs/client/ca.pem
periodSeconds: 1
resources:
requests:
memory: 512Mi
securityContext:
privileged: true
volumeMounts:
- mountPath: /certs/client
name: dind-certs
steps:
- args:
- '--storage-driver=vfs'
- '--debug'
env:
- name: DOCKER_HOST
value: 'tcp://localhost:2376'
- name: DOCKER_TLS_VERIFY
value: '1'
- name: DOCKER_CERT_PATH
value: /certs/client
- name: DOCKER_USER
valueFrom:
secretKeyRef:
key: username
name: docker-auth
- name: DOCKER_TOKEN
valueFrom:
secretKeyRef:
key: password
name: docker-auth
- name: DIND_CONFIG
valueFrom:
configMapKeyRef:
key: file
name: dind-env
image: docker
name: docker-build
resources: {}
script: |
#!/usr/bin/env sh
set -e
pwd
ls -ltr /workspace/source
docker run --privileged -v "/workspace/source:/workspace" busybox ls -ltr /workspace
securityContext:
privileged: true
volumeMounts:
- mountPath: /certs/client
name: dind-certs
workingDir: $(workspaces.source.path)
volumes:
- emptyDir: {}
name: dind-certs
workspaces:
- name: source
basically we have to attach volume to sidecar, since docker run happens in side card
volumeMounts:
- mountPath: /certs/client
name: dind-certs
- name: $(workspaces.source.volume)
mountPath: $(workspaces.source.path)

Drone template not triggering build

Following is how our.drone.yml looks like (and template also listed below) this an example configuration very much how we want in our production. The reason we are using a template is that our staging and production have similar configurations with values different in them(hence circuit template). And we wanted to remove duplication using the template circuit.yaml.
But currently, we are unable to do so df I don’t define the test.yaml(template) and have test step imported without template (and have the circuit template define to avoid the duplicate declaration of staging and production build) the drone build fails with
"template converter: template name given not found
If I define the test step as a template. I see the test step working but on creating the tag I see the following error
{"commit":"28ac7ad3a01728bd1e9ec2992fee36fae4b7c117","event":"tag","level":"info","msg":"trigger: skipping build, no matching pipelines","pipeline":"test","ref":"refs/tags/v1.4.0","repo":"meetme2meat/drone-example","time":"2022-01-07T19:16:15+05:30"}
---
kind: template
load: test.yaml
data:
commands:
- echo "machine github.com login $${GITHUB_LOGIN} password $${GITHUB_PASSWORD}" > /root/.netrc
- chmod 600 /root/.netrc
- go clean -testcache
- echo "Running test"
- go test -race ./...
---
kind: template
load: circuit.yaml
data:
deploy: deploy
create_tags:
commands:
- echo "Deploying version $DRONE_SEMVER"
- echo -n "$DRONE_SEMVER,latest" > .tags
backend_image:
version: ${DRONE_SEMVER}
tags:
- '${DRONE_SEMVER}'
- latest
And the template is below
test.yaml
kind: pipeline
type: docker
name: test
steps:
- name: test
image: golang:latest
environment:
GITHUB_LOGIN:
from_secret: github_username
GITHUB_PASSWORD:
from_secret: github_token
commands:
{{range .input.commands }}
- {{ . }}
{{end}}
volumes:
- name: deps
path: /go
- name: build
image: golang:alpine
commands:
- go build -v -o out .
volumes:
- name: deps
path: /go
volumes:
- name: deps
temp: {}
trigger:
branch:
- main
event:
- push
- pull_request
circuit.yaml
kind: pipeline
type: docker
name: {{ .input.deploy }}
steps:
- name: create-tags
image: alpine
commands:
{{range .input.create_tags.commands }}
- {{ . }}
{{end}}
- name: build
image: plugins/docker
environment:
GITHUB_LOGIN:
from_secret: github_username
GITHUB_PASSWORD:
from_secret: github_token
VERSION: {{ .input.backend_image.version }}
SERVICE: circuits
settings:
auto_tag: false
repo: ghcr.io/meetme2meat/drone-ci-example
registry: ghcr.io

Tekton trigger flow from github

I am learning Tekton (for business), coming from github actions (private).
The Tekton docs (or any other tutorial I could find) have instructions on how to automatically start a pipeline from a github push. Basically they all somewhat follow the below flow: (I am aware of PipelineRun/TaskRun etc)
Eventlistener - Trigger - TriggerTemplate - Pipeline
All above steps are basically configuration steps you need to take (and files to create and maintain), one easier than the other but as far as I can see they also need to be taken for every single repo you're maintaining. Compared to github actions where I just need 1 file in my repo describing everything I need this seems very elaborate (if not cumbersome).
Am I missing something ? Or is this just the way to go ?
Thanks !
they also need to be taken for every single repo you're maintaining
You're mistaken here.
The EventListener receives the payload of your webhook.
Based on your TriggerBinding, you may map fields from that GitHub payload, to variables, such as your input repository name/URL, a branch or ref to work with, ...
For GitHub push events, one way to do it would be with a TriggerBinding such as the following:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerBinding
metadata:
name: github-push
spec:
params:
- name: gitbranch
value: $(extensions.branch_name) # uses CEL interceptor, see EL below
- name: gitrevision
value: $(body.after) # uses body from webhook payload
- name: gitrepositoryname
value: $(body.repository.name)
- name: gitrepositoryurl
value: $(body.repository.clone_url)
We may re-use those params within our TriggerTemplate, passing them to our Pipelines / Tasks:
apiVersion: triggers.tekton.dev/v1alpha1
kind: TriggerTemplate
metadata:
name: github-pipelinerun
spec:
params:
- name: gitbranch
- name: gitrevision
- name: gitrepositoryname
- name: gitrepositoryurl
resourcetemplates:
- apiVersion: tekton.dev/v1beta1
kind: PipelineRun
metadata:
generateName: github-job-
spec:
params:
- name: identifier
value: "demo-$(tt.params.gitrevision)"
pipelineRef:
name: ci-docker-build
resources:
- name: app-git
resourceSpec:
type: git
params:
- name: revision
value: $(tt.params.gitrevision)
- name: url
value: $(tt.params.gitrepositoryurl)
- name: ci-image
resourceSpec:
type: image
params:
- name: url
value: registry.registry.svc.cluster.local:5000/ci/$(tt.params.gitrepositoryname):$(tt.params.gitrevision)
- name: target-image
resourceSpec:
type: image
params:
- name: url
value: registry.registry.svc.cluster.local:5000/ci/$(tt.params.gitrepositoryname):$(tt.params.gitbranch)
timeout: 2h0m0s
Using the following EventListener:
apiVersion: triggers.tekton.dev/v1alpha1
kind: EventListener
metadata:
name: github-listener
spec:
triggers:
- name: github-push-listener
interceptors:
- name: GitHub push payload check
github:
secretRef:
secretName: github-secret # a Secret you would create (option)
secretKey: secretToken # the secretToken in my Secret matches to secret configured in GitHub, for my webhook
eventTypes:
- push
- name: CEL extracts branch name
ref:
name: cel
params:
- name: overlays
value:
- key: truncated_sha
expression: "body.after.truncate(7)"
- key: branch_name
expression: "body.ref.split('/')[2]"
bindings:
- ref: github-push
template:
ref: github-pipelinerun
And now, you can expose that EventListener, with an Ingress, to receive notifications from any of your GitHub repository.

Tekton - mount path workspace issue - Error of path

Currently, I am trying to deploy tutum-hello-world. I have written a script for the same, but it does not work as it is supposed to.
I am certain that this issue is related to workspace.
UPDATE
Here is my code for task-tutum-deploy.yaml-
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: tutum-deploy
spec:
steps:
- name: tutum-deploy
image: bitnami/kubectl
script: |
kubectl apply -f /root/tekton-scripts/tutum-deploy.yaml
workspaces:
- name: messages
optional: true
mountPath: /root/tekton-scripts/
Error -
root#master1:~/tekton-scripts# tkn taskrun logs tutum-deploy-run-8sq8s -f -n default
[tutum-deploy] + kubectl apply -f /root/tekton-scripts/tutum-deploy.yaml
[tutum-deploy] error: the path "/root/tekton-scripts/tutum-deploy.yaml" cannot be accessed: stat /root/tekton-scripts/tutum-deploy.yaml: permission denied
container step-tutum-deploy has failed : [{"key":"StartedAt","value":"2021-06-14T12:54:01.096Z","type":"InternalTektonResult"}]
PS - I have placed my script on the master node at - /root/tekton-scripts/tutum-deploy.yaml
root#master1:~/tekton-scripts# ls -l tutum-deploy.yaml
-rwxrwxrwx 1 root root 626 Jun 11 11:31 tutum-deploy.yaml
OLD SCRIPT
Here is my code for task-tutum-deploy.yaml-
apiVersion: tekton.dev/v1beta1
kind: Task
metadata:
name: tutum-deploy
spec:
workspaces:
- name: messages
optional: true
mountPath: /root/tekton-scripts/tutum-deploy.yaml
steps:
- name: tutum-deploy
image: bitnami/kubectl
command: ["kubectl"]
args:
- "apply"
- "-f"
- "./tutum-deploy.yaml"
Here is my code for tutum-deploy.yaml which is present on the machine (master node) of Kubernetes cluster with read, write and execute permissions -
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-tutum
labels:
service: hello-world-tutum
spec:
replicas: 1
selector:
matchLabels:
service: hello-world-tutum
template:
metadata:
labels:
service: hello-world-tutum
spec:
containers:
- name: tutum-hello-world
image: tutum/hello-world:latest
ports:
- containerPort: 80
---
apiVersion: v1
kind: Service
metadata:
name: hello-world-tutum
spec:
type: NodePort
selector:
service: hello-world-tutum
ports:
- name: "80"
port: 80
targetPort: 80
nodePort: 30050
I ran the following commands from my master node of Kubernetes cluster -
1. kubectl apply -f task-tutum-deploy.yaml
2. tkn task start tutum-deploy
Error -
Using tekton command - $ tkn taskrun logs tutum-deploy-run-tvlll -f -n default
task tutum-deploy has failed: "step-tutum-deploy" exited with code 1 (image: "docker-pullable://bitnami/kubectl#sha256:b83299ee1d8657ab30fb7b7925b42a12c613e37609d2b4493b4b27b057c21d0f"); for logs run: kubectl -n default logs tutum-deploy-run-tvlll-pod-vbl5g -c step-tutum-deploy
[tutum-deploy] error: the path "./tutum-deploy.yaml" does not exist
container step-tutum-deploy has failed : [{"key":"StartedAt","value":"2021-06-11T14:01:49.786Z","type":"InternalTektonResult"}]
The error is from this part of your YAML:
spec:
workspaces:
- name: messages
optional: true
mountPath: /root/tekton-scripts/tutum-deploy.yaml
spec.workspaces.mountPath expects a directory, rather than a file, as you have specified here. You may mean /root/tekton-scripts/ instead but I am unfamiliar with tutum-hello-world.
If you look at the documentation you will see that all references to mountPath are directories rather than files.