We would like to disable http access on our GitLab instance and use SSH only. Can drone somehow communicate with GitLab over SSH?
The default clone plugin uses git+https to clone repositories. If you would like to change the default behavior and use git+ssh, you will have to create a custom clone plugin.
clone:
custom:
image: amazia/custom-git-plugin
pipeline:
build:
image: golang
commands:
- go build
- go test
The above example demonstrates a yaml configuration that overrides the default clone step to use a custom plugin. Here are some resources for creating custom plugins:
http://docs.drone.io/creating-custom-plugins-bash/
http://docs.drone.io/creating-custom-plugins-golang/
Related
I'd like to be able to start my pipeline which uses two workspaces using the Tekton dashboard (GUI) but from what I can see, there is no option to provide workspaces here.
The only two options of creating a PipelineRun with bound workspaces I can think of are to either:
Create it programmatically and apply using either kubectl or tekton-cli
Create a TriggerTemplate with bound workspaces and run the pipeline by a webhook to an EventListener
My main problem is both of those options require the developer to go through a very non-user-friendly process. Ideally, I'd like to run the pipelines with bound workspaces from tekton GUI. Can this be somehow achieved?
I've tried providing the binding in the workspaces section of the Pipeline definition as below:
workspaces:
- name: source-dir
persistentVolumeClaim:
claimName: gradle-cache-pvc
- name: ssh-creds
secret:
secretName: ssh-key-secret
I have a Kubernetes cluster on Azure, where I use Helm to make it easier to manage micro-services and other tools on it, and Keycloak is one of them.
I need to use magic link authenticator in one of my apps, I'm aware that I need to add an extension in my Keycloak chart, but I don't know how.
In the image repository I'm using, they explain how to add custom themes, via extraInitContainers param on chart configuration. I think I can achieve what I want through it.
In this tutorial they say that's the extension, but I have no idea how to add this to my Keycloak instance on k8s by using helm charts. How do I achieve that?
Just more info about my config, I'm running louketo-proxy(as a side car) on some apps where I want to protect.
To publish a theme with original image, first create an archive with the thema.
Create a file custom-themes-values.yml with a content:
extraInitContainers: |
- name: theme-provider
image: busybox
imagePullPolicy: IfNotPresent
command:
- sh
args:
- -c
- |
echo "wgetting theme from maven..."
wget -O /theme/keycloak-theme.jar https://repo1.maven.org/maven2/org/acme/keycloak-theme/1.0.0/keycloak-theme-1.0.0.jar
volumeMounts:
- name: theme
mountPath: /theme
extraVolumeMounts: |
- name: theme
mountPath: /opt/jboss/keycloak/standalone/deployments
extraVolumes: |
- name: theme
emptyDir: {}
Run with:
helm install keycloak codecentric/keycloak --values custom-themes-values.yml
ps: This example the theme was publish into maven repository, but you can copy a local file to.
This way you can adapt to magic-link.
I am using Github as my source control, and using Github Actions as my CI/CD solution. I have a dedicated Windows Server somewhere, which accepts the published version of my repo.
I have a Github Action, which does Built, Test, Publish and Deploy (using FTP). I am not convinced with the "Deploy" idea. for example lets say my website has a huge codebase, and then FTPing everytime when we make a commit (Push) is NOT really a productive idea (i am publishing the published directory...not the Source Code to my server). Sometimes FTPing simply does not work, due to IIS Locking the files. what is the most reliable way to publish/deploy files to a remote server using Github Actions (or any other provider).
below is my sample Yaml file...
name: .NET Core
on:
push:
branches: [ master ]
pull_request:
branches: [ master ]
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v2
- name: Setup .NET Core
uses: actions/setup-dotnet#v1
with:
dotnet-version: 2.1
- name: Install dependencies
run: dotnet restore
- name: Build
run: dotnet build --configuration Release --no-restore
- name: Test
run: dotnet test --no-restore --verbosity normal
- name: Run the Publish
run: dotnet publish --configuration Release --no-restore --output ./publish
- name: FTP-Deploy-Action
uses: sebastianpopp/ftp-action#releases/v2
with:
host: my_host
user: my_user
password: ${{ secrets.FTP_PASSWORD }}
localDir: "/home/runner/work/CiTest/CiTest/CiTest/publish"
remoteDir: "CI-Test"
I wanted to add an answer to this question because Rosco's answer helped me, but I have more to contribute.
We've set up a self-hosted runner on the internal IIS server we want to deploy to. So our GitHub Actions flow looks a bit like yours to begin with (up to the dotnet publish command), but then has some extra bits.
First, still in the cloud, we "upload" artifacts from the build (I'm replacing our project name with "Foo"):
- name: Upload a Build Artifact
uses: actions/upload-artifact#v2.2.1
with:
name: application
path: /home/runner/work/Foo/Foo/Foo/bin/Release/net5.0/publish/
Then we have a whole new job that runs on prem:
deploy:
needs: build
runs-on: self-hosted
steps:
- name: Take application offline
run: New-Item -Type File -Name app_offline.htm -Path E:\Foo -Force
- name: Download new binaries over the top of the app
uses: actions/download-artifact#v2
with:
name: application
path: E:\Foo
- name: Bring the app back online
run: Remove-Item E:\Foo\app_offline.htm
So this creates an app_offline.htm file in our website folder (E:\Foo), then pulls the artifacts down from GitHub directly into that folder. When it's done, it deletes the app_offline.htm file and the site starts again.
It works a treat! Obviously keen to hear feedback from others if you think I'm doing something wrong, but I wanted to get that up here because this question ranks pretty highly when you search for "github actions deploy iis".
Since you have a dedicated server for your application, I would recommend using a Github self-hosted runner.
You install the runner service on your server and then the Github CI/CD build process runs on your dedicated server.
The build script can build your app then stop the IIS site locally which prevents locked files.
After that it can copy (or robocopy) the new build into the app folder.
Gitlab also has self-hosted runners.
i'm trying to implement automation inside my GitLab project.
In order to perform security scan, i would like to use ZAP to go through all the URLs present in the
project and scan them. It's clearly not possible to pass manually all the URLs, so i'm trying to find a way to make all the test as automated as possible.
The problem is: how to reach all the URLs present in the application?
I thought a way could be to pass them as a "variable" in the YML file, and use them as parameter in the ZAP command, something like that (see below).
Is this a reasonable solution? Is there any other way to perform an automated scan inside a repository (without passing manually the URLs)?
Thanks
variables:
OWASP_CONTAINER: $APP_NAME-$BUILD_ID-OWASP
OWASP_IMAGE: "owasp/zap2docker-stable"
OWASP_REPORT_DIR: "owasp-data"
ZAP_API_PORT: "8090"
PENTEST_IP: 'application:8080'
run penetration tests:
stage: pen-tests
image: docker:stable
- docker exec $OWASP_CONTAINER zap-cli -v -p $ZAP_API_PORT active-scan http://$PENTEST_IP/html
You need to turn on a new feature flag (FF_NETWORK_PER_BUILD) to enable a network per build. Then also services can reach each others (Available since GitLab runner 12.9). For more information see: https://docs.gitlab.com/runner/executors/docker.html#networking
Working example owasp zap job in GitLab CI:
owasp-zap:
variables:
FF_NETWORK_PER_BUILD: 1
image: maven
services:
- selenium/standalone-chrome
- name: owasp/zap2docker-weekly
entrypoint: ['zap.sh', '-daemon', '-host', '0.0.0.0', '-port', '8080',
'-config', 'api.addrs.addr.name=.*', '-config', 'api.addrs.addr.regex=true', '-config', 'api.key=1234567890']
script:
- sleep 5
- mvn clean test -Dbrowser=chrome -Dgrid_url=http://selenium-standalone-chrome:4444/wd/hub -Dproxy=http://owasp-zap2docker-weekly:8080
- curl http://owasp-zap2docker-weekly:8080/OTHER/core/other/htmlreport/?apikey=1234567890 -o report.html
artifacts:
paths:
- report.html
Having a Spring Boot application we are using application.yml file to store properties. I got a task to give a user a possibility to override some properties while starting an application. Taking into consideration we have dockerised our app docker-compose file is the very right place I believe for that. I found one option which works actually, env_file:
backend:
build:
context: backend
dockerfile: Dockerfile.backend
restart: always
ports:
- 3000:3000
env_file:
- backend/custom.env
volumes:
- ../m2_repo:/root/.m2/
- ../{APP_NAME}/data_sources:/backend/data_sources/
links:
- database
networks:
main:
aliases:
- backend
This solves perfectly my task and all the KEY=VALUE pairs override existing in application.yml properties. However, I have 2 questions:
It appeared that having multiple services in my docker-compose file I need specify a separate env_file for each service, which is probably not very convenient. Is there a possibility to have one common env_file for the whole docker-compose file?
I know that for docker-compose run command there is an option -e where i can put key=value pairs of env variables. Is there any similar option for docker-compose up? I mean in order not to use env_file at all.
Ad 1: It is not possible. I also believe it is intentional - to make the developer define what container has access to what .env data.
Ad 2: No, you cannot supply the variables using a runtime parameter of up command of docker-compose (run docker-compose help up to see the available runtime params). But you can define these using environment clause from within a compose file, like:
restart: always
ports:
- 3000:3000
env_file:
- backend/custom.env
environment:
- DB_PASSWORD # <= #1
- APP_ENV=production # <= #2
ie.
either just a name of the env var - its value is then taken from the host machine
or the whole definition to create a new one to be available within a container
See docs on environment clause for more clarification.
Another thing you can do in order to override some settings is to extend the compose file using a "parent" one. Docs on extends clause
Unfortunately as of now, extends won't work when using compose file of version 3, but it is being discussed in this github issue, so hopefully it will be available soon:)