Here is a github workflow that I would like to use to create a release.
name: make release
on:
workflow_dispatch:
jobs:
try-to-create-a-tag:
runs-on: ubuntu-latest
steps:
- name: Create release
run: curl \
-X POST \
-H "Accept:application/vnd.github+json" \
-H "Authorization:token ${ secrets.GITHUB_TOKEN }" \
https://api.github.com/repos/.../releases \
-d '{"tag_name":"tag_test11","target_commitish":"main","name":"tag_test11","body":"Description of the release","draft":false,"prerelease":false,"generate_release_notes":false}'
The workflow runs without error, however I get the following message from curl:
authorization:token ${ secrets.GITHUB_TOKEN }: bad substitution
I have also tried using double braces, however this didn't work either. How can I substitute the variable into the curl command?
NOTE: the error is that one must use a | right after run: when splitting over multiple lines.
You have to use double brackets:
${{ secrets.GITHUB_TOKEN }}
Documentation:
https://docs.github.com/en/actions/security-guides/encrypted-secrets#example-using-bash
jobs:
try-to-create-a-tag:
runs-on: ubuntu-latest
steps:
- name: Create release
run: |
curl \
-X POST \
-H "Accept:application/vnd.github+json" \
-H "Authorization:token ${{ secrets.GITHUB_TOKEN }}" \
https://api.github.com/repos/.../releases \
-d '{"tag_name":"tag_test11","target_commitish":"main","name":"tag_test11","body":"Description of the release","draft":false,"prerelease":false,"generate_release_notes":false}'
Related
in my Github action I use a temporary variable for a timestamp and want to send this variable in a webhook, but github cant interpret the $var in string. Can someone pls help me?
The code looks like this:
- name: Posting Rocketchat
if: failure()
run: |
TS=$(date +%Y%m%d%H%M%S)
curl -X POST -H 'Content-Type: application/json' --data '{"text":"❌ Test: Leads Testing Desktop 💻","image_url":"https://myurl.net/$TS-1.png"}' https://chat.myurl.com/hooks/yxyxyxxyxyxyx/xxxxxyxyxyxyxyxyxy
Use instead
- name: Posting Rocketchat
if: failure()
run: |
echo "TS=$(date +%Y%m%d%H%M%S)" >> $GITHUB_ENV
curl -X POST -H 'Content-Type: application/json' --data '{"text":"❌ Test: Leads Testing Desktop 💻","image_url":"https://myurl.net/${{ env.TS }}-1.png"}' https://chat.myurl.com/hooks/yxyxyxxyxyxyx/xxxxxyxyxyxyxyxyxy
I want to do a zap full scan on gitlab cicd with authentication to the website i want to run it (without the DAST module from gitlab)
i can run the zap-full-scan.py properly but dont know how to add authentication credentials for the site
stages:
- scan
dast:
stage: scan
image:
name: owasp/zap2docker-weekly
before_script:
- mkdir -p /zap/wrk
script:
- pwd
- ls
- zap-full-scan.py -t "http://example.com" -m 1 -d -I -r testreport.html
- cp /zap/wrk/testreport.html testreport.html
artifacts:
when: always
paths:
- testreport.html
It's also possible to authenticate the user before performing DAST checks:
dast:
image: registry.gitlab.com/gitlab-org/security-products/zaproxy
variables:
website: "https://example.com"
login_url: "https://example.com/sign-in"
script:
- mkdir /zap/wrk/
- /zap/zap-baseline.py -J gl-dast-report.json -t $website \
--auth-url $login_url \
--auth-username "john.doe#example.com" \
--auth-password "john-doe-password" || true
- cp /zap/wrk/gl-dast-report.json .
artifacts:
paths: [gl-dast-report.json]
See zaproxy documentation to learn more about authentication settings.
using this modified version https://github.com/ICTU/zap2docker-auth-weekly
stages:
- scan
dast:
stage: scan
image:
name: ictu/zap2docker-weekly
before_script:
- mkdir -p /zap/wrk
script:
- pwd
- ls
- zap-full-scan.py -t "http://testphp.vulnweb.com" -I -r testreport.html --hook=/zap/auth_hook.py -z "auth.loginurl=http://example.com/login.php auth.username_field="uname" auth.password_field="pass" auth.username="username" auth.password="pass""
- cp /zap/wrk/testreport.html testreport.html
artifacts:
when: always
paths:
- testreport.html
Run ZAP locally and get authentication working as per https://www.zaproxy.org/docs/authentication/
Then export your context file and specify that and the user you want to use as per https://www.zaproxy.org/docs/docker/full-scan/
Dependencies in package.json:
"dependencies": {
"my-repo": "git+ssh://github.com/org-name/my-repo.git"
},
GitHub Actions:
name: Test
on: [push, pull_request]
jobs:
test:
name: Test
runs-on: ubuntu-18.04
steps:
- name: Checkout
uses: actions/checkout#v2
- name: Setup Node.js
uses: actions/setup-node#v2
with:
node-version: 12
registry-url: 'https://npm.pkg.github.com'
scope: '#org-name'
- uses: webfactory/ssh-agent#v0.4.1
with:
ssh-private-key: ${{ secrets.DEPLOY_KEY }}
- name: Install dependencies
run: yarn
- name: Test
run: yarn test
The DEPLOY_KEY in the GitHub Actions secrets is the private key and I've added the corresponding public key as a deploy key in the dependency repo.
I generated the key with ssh-keygen -m PEM -t rsa -b 4096 -C "ssh://github.com/org-name/my-repo.git" -f ./deploykey -q -N ""
Here's the failure I see in the GitHub Actions output:
Exit code: 128
Command: git
Arguments: ls-remote --tags --heads ssh://github.com/org-name/my-repo.git
Directory: /home/runner/work/auth-package/auth-package
Output:
Warning: Permanently added the RSA host key for IP address '140.82.112.4' to the list of known hosts.
runner#github.com: Permission denied (publickey).
fatal: Could not read from remote repository.
Please make sure you have the correct access rights
and the repository exists.
Really losing my mind on this one!
I don't know If I'm getting it right but to access to a different repo you need an access token. Therefore you need a Github Application with a private key secret. https://github.com/settings/apps
You need those three env variables in your workflow:
- uses: actions/checkout#v2
- name: get secrets
env:
PRIVATE_KEY: ${{ secrets.PRIVATE_KEY }}
APP_ID: ${{ secrets.APP_ID }}
INSTALLATION_ID: ${{ secrets.INSTALLATION_ID }}
Then create your JWT (check your JWT here: https://jwt.io/) to create the access token via REST API
run: |
PEM=$PRIVATE_KEY
GITHUB_APP_ID=$APP_ID
NOW=$( date +%s )
IAT="${NOW}"
EXP=$((${NOW} + 600))
HEADER_RAW='{"alg":"RS256"}'
HEADER=$( echo -n "${HEADER_RAW}" | openssl base64 | tr -d '=' | tr '/+' '_-' | tr -d '\n' )
PAYLOAD_RAW='{"iat":'"${IAT}"',"exp":'"${EXP}"',"iss":'"${GITHUB_APP_ID}"'}'
PAYLOAD=$( echo -n "${PAYLOAD_RAW}" | openssl base64 | tr -d '=' | tr '/+' '_-' | tr -d '\n' )
HEADER_PAYLOAD="${HEADER}"."${PAYLOAD}"
SIGNATURE=$( openssl dgst -sha256 -sign <(echo -n "${PEM}") <(echo -n "${HEADER_PAYLOAD}") | openssl base64 | tr -d '=' | tr '/+' '_-' | tr -d '\n' )
JWT="${HEADER_PAYLOAD}"."${SIGNATURE}"
Then start your API CALL:
ACCESS_TOKEN=$(curl -sS -X POST \
-H "Authorization: Bearer "$JWT"" \
-H "Accept: application/vnd.github.v3+json" \
https://github.com/api/v3/app/installations/"$INSTALLATION_ID"/access_tokens | grep -o '"token": "[^"]*' | grep -o '[^"]*$')
Then start with your Git Clone command:
git clone https://x-access-token:"$ACCESS_TOKEN"#github.com/../repo.git
cd repo
git config --global user.email "<email>"
git config --global user.name "<name>"
git branch upload
git checkout upload
git commit -m "update"
git push --set-upstream origin upload
Then you can clone other repos in your current one for example. Be aware to grant the app access to the repos.
I am a beginner with Rundeck, I would like to create a job to test logins in a specific website.
How do I do that?
You can create a job with an inline script step that calls the site API using CURL.
For example, to test against rundeck itself you can use the following script inside a rundeck job:
curl -v -c cookie -b cookie -d j_username=admin -d j_password=admin http://localhost:4440/j_security_check \
-H "Accept: application/json" \
http://localhost:4440/api/38/system/info/
And the job definition, you can copy it and save it as a YAML file, then import it to your project to test.
- defaultTab: nodes
description: ''
executionEnabled: true
id: 2dc15b18-5b2c-4081-be72-17eb3618d68c
loglevel: INFO
name: Login
nodeFilterEditable: false
plugins:
ExecutionLifecycle: null
scheduleEnabled: true
sequence:
commands:
- fileExtension: .sh
interpreterArgsQuoted: false
script: |-
curl -v -c cookie -b cookie -d j_username=admin -d j_password=admin http://localhost:4440/j_security_check \
-H "Accept: application/json" \
http://localhost:4440/api/38/system/info/ | jq
scriptInterpreter: /bin/bash
keepgoing: false
strategy: node-first
uuid: 2dc15b18-5b2c-4081-be72-17eb3618d68c
I am using rest API to run manual jobs in GitLab CI. When i start a manual job from UI I am able to define custom variables that i can use during the job. How can i define them when running job through API?
Could not find any documentation on it. Or not even a single question in forums.
This is how i currently run my job
curl -k --request POST --header "PRIVATE-TOKEN: abc" https://mygit.com/api/v4/projects/17/jobs/1956/play
I tried adding:
--form variables[TEST]=hello
But this didnt work.
Edit:
A bit more information on what im doing. So my pipeline has two stages. Build and deploy. On each commit I want build to run once and then i want to be able to deploy this result to multiple different servers. Because the server list is dynamic and there are a lot of them I want to have the IP address of the server as an variable I can give to my deploy job.
Instead of starting a job you can start a pipeline and set the variables from there. Here's an example of how to do this from the GitLab documentation:
curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
--header "Content-Type: application/json" \
--data '{ "ref": "master", "variables": [ {"key": "VAR1", "value": "hello"}, {"key": "VAR2", "value": "world"} ] }' \
"https://gitlab.example.com/api/v4/projects/169/pipeline"
This is the way how I'm using it, didn't find a way to use API tokens for it though.
curl -X POST \
-F token=xxxxxxxxxxxxxxxx \
-F "ref=some_branch" \
-F "variables[VAR1]=abc" \
-F "variables[VAR2]=cde" \
"https://example.gitlab.com/api/v4/projects/312/trigger/pipeline"
Where -F "variables[VAR1]=abc" for example is set in .gitlab-ci.yml.
only:
variables:
- $VAR1
The idea was to create some manual CI jobs and tell the devs they can run them via API call, but since I can only use the project token here, it's absolutely not secure.
It would be really handy to run it via
curl --request PUT --header "PRIVATE-TOKEN: <your_access_token>"
Passing variables is documented in gitlab-org/gitlab issue 2772, but more about triggering pipeline (not job)
See if that syntax would work, for trigger variables (syntax variables[xxx]=yyy):
# gitlab-ci.yml
build:
script:
- curl --request POST --form "variables[PRE_CI_PIPELINE_SOURCE]=$CI_PIPELINE_SOURCE" --form "token=$CI_JOB_TOKEN" --form ref=master http://192.168.10.3:3001/api/v4/projects/13/trigger/pipeline
Or simply for regular variables --form key=value:
curl --request POST --form "token=$CI_JOB_TOKEN" --form ref=master https://gitlab.example.com/api/v4/projects/9/trigger/pipeline
It looks like that as of Jan 25, 2021 this feature not yet supported. There is a feature request I found here: https://gitlab.com/gitlab-org/gitlab/-/issues/37267
Update 2022-03:
After you create a trigger token, and create trigger_pipeline step in pipeline, like this
trigger_pipeline:
tags:
image: alpine:latest
stage: deploy
script:
only:
variables:
- $MANUAL
you can use it to trigger pipelines with a tool that can access the API
curl --request POST \
--form token=TOKEN \
--form ref=main \
--form "variables[MANUAL]=true" \
"https://gitlab.example.com/api/v4/projects/123456/trigger/pipeline"
or a webhook:
https://gitlab.example.com/api/v4/projects/123456/ref/<ref_name>/trigger/pipeline?token=<token>
for example for manual run.