How do I access the root of the GitHub Repository in a GitHub action? - pdf

I am trying to build a resume pdf from a resume.tex in my resume repository file whenever a commit is made.
I am getting the following error.
Error: File '/home/runner/work/resume/resume/resume.tex' cannot be found from the directory '/github/workspace'.
How should I access the resume.tex file? If I just say root_file: resume.tex, the error is:
Error: File 'resume.tex' cannot be found from the directory '/github/workspace'.
The .github/workflows/build_resume.yml file looks like this. The resume.tex file is in the root of my repository.
# This is a basic workflow to help you get started with Actions
name: Build PDF on commit.
# Controls when the workflow will run
on:
# Triggers the workflow on push or pull request events but only for the main branch
push:
branches: [ main ]
# Allows you to run this workflow manually from the Actions tab
# workflow_dispatch:
# A workflow run is made up of one or more jobs that can run sequentially or in parallel
jobs:
# This workflow contains a single job called "build"
build:
# The type of runner that the job will run on
runs-on: ubuntu-latest
# Steps represent a sequence of tasks that will be executed as part of the job
steps:
- name: Github Action for LaTeX
uses: xu-cheng/latex-action#v2
with:
# The root LaTeX file to be compiled
root_file: ${{ github.workspace }}/resume.tex
# Interpret the root_file input as bash glob pattern
# glob_root_file: # optional
# The working directory for this action
# working_directory: # optional
# The LaTeX engine to be invoked
# compiler: # optional, default is latexmk
# Extra arguments to be passed to the LaTeX engine
# args: # optional, default is -pdf -file-line-error -halt-on-error -interaction=nonstopmode
# [Deprecated] Install extra packages by tlmgr
# extra_packages: # optional
# Install extra packages by apk
# extra_system_packages: # optional
# Install extra .ttf/.otf fonts.
# extra_fonts: ./fonts/*.ttf
# Arbitrary bash codes to be executed before compiling LaTeX documents
pre_compile: tlmgr update --self && tlmgr update --all
# Arbitrary bash codes to be executed after compiling LaTeX documents
post_compile: latexmk -c
# Instruct latexmk to enable --shell-escape
# latexmk_shell_escape: # optional
# Instruct latexmk to use LuaLaTeX
# latexmk_use_lualatex: # optional
# Instruct latexmk to use XeLaTeX
latexmk_use_xelatex: true

When you want to execute files from the current repository, you need to use the actions/checkout first (at the beginning of your job's steps).
This will allow you to access the repository $github_workspace (one of Github environment variables) in your workflow.
Note: All commands you'll run after using the action/checkout will be executed at the repository root.
For example, considering that your resume.tex file is at the root of the repository, you would use something like this:
name: Example
on:
push:
pull_request:
jobs:
build:
runs-on: ubuntu-latest
steps:
- name: checkout repo
uses: actions/checkout#v2.3.4
- name: show resume.tex file content
run: cat resume.tex
Here is another workflow example from a personal repository, following the same logic if you want to execute a specific script located in the repository in your workflow. to perform any operation.

If someone is looking to do the same, the following steps need to be done.
Go into your repository.
Compile latex document using xu-cheng/latex-action#v2
Check if PDF is generated.
Push PDF to repository.
The configuration file to do that will look like this.
name: Github Actions CI to build pdf from tex source.
on: push
jobs:
build:
if: "!contains(github.event.head_commit.message, '[skip ci]')"
runs-on: ubuntu-latest
steps:
- name: Set up Git repository
uses: actions/checkout#v2
- name: Compile LaTeX document
uses: xu-cheng/latex-action#v2
with:
# The root LaTeX file to be compiled
root_file: resume_aditya_wagh.tex
# Interpret the root_file input as bash glob pattern
# glob_root_file: # optional
# The working directory for this action
# working_directory: # optional
# The LaTeX engine to be invoked
# compiler: # optional, default is latexmk
# Extra arguments to be passed to the LaTeX engine
args: -pdf -file-line-error -halt-on-error -interaction=nonstopmode
# Install extra packages by apk
# extra_system_packages: # optional
# Install extra .ttf/.otf fonts.
extra_fonts: ./fonts/*.ttf
# Arbitrary bash codes to be executed before compiling LaTeX documents
pre_compile: tlmgr update --self && tlmgr update --all
# Arbitrary bash codes to be executed after compiling LaTeX documents
post_compile: latexmk -c
# Instruct latexmk to enable --shell-escape
# latexmk_shell_escape: # optional
# Instruct latexmk to use LuaLaTeX
# latexmk_use_lualatex: # optional
# Instruct latexmk to use XeLaTeX
latexmk_use_xelatex: true
- name: Check pdf file
run: |
file resume_aditya_wagh.pdf | grep -q ' PDF '
- name: Upload file to repository
run: |
git config --global user.name "adityamwagh"
git config --global user.email "adityamwagh#gmail.com"
git add resume_aditya_wagh.pdf
git commit -m "commit message"
git push
if: github.event_name == 'push' && github.ref == 'refs/heads/main'

Related

Can github actions edit parts of README.md?

I want to update ONLY the first header of a readme so it always has the repo's name.
I know how to get the repo name in github actions by doing something like:
name: CI
on:
push:
jobs:
build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout#v3
- name: Run a one-line script
run: echo "REPO_NAME=${{ github.event.repository.name }}" >> $GITHUB_ENV
However, I want to access my readme.md and add the 'github.event.repository.name' to the header.
So I would make a Readme with something like:
Introduction for: {{ github.event.repository.name }}
hoping I can get something like this with gitactions:
Introduction for: RepoName
I tried using some marketplace github actions but the one I tried seems to not do variables and it seems to update the whole readme.md not just the header: https://github.com/marketplace/actions/dynamic-readme
Here is the failed example for this marketplace plugin:
name: Dynamic Template
on:
push:
branches:
- main
workflow_dispatch:
jobs:
update_templates:
name: "Update Templates"
runs-on: ubuntu-latest
steps:
- name: "📥 Fetching Repository Contents"
uses: actions/checkout#main
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo "REPO_NAME=${{ github.event.repository.name }}" >> $GITHUB_ENV
# Runs a set of commands using the runners shell
- name: Update GitHub Profile README
uses: theboi/github-update-readme#v1.0
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
REPO_NAME: ${{ github.event.repository.name }}
with:
header: $REPO_NAME
Is there any way to make the readme.md file have the repo name dynamically with a variable in github actions?
EDIT: I think I am very close but I don't know how to commit code in github actions. I figured, I can do this manually by using the sed command in bash in github actions. I think it worked but I think I have to commit the code to make it save. Here is the code I have so far:
name: Dynamic Template
on:
push:
branches:
- main
workflow_dispatch:
jobs:
update_templates:
name: "Update Templates"
runs-on: ubuntu-latest
steps:
- name: "📥 Fetching Repository Contents"
uses: actions/checkout#main
# Runs a single command using the runners shell
- name: Run a one-line script
run: echo "REPO_NAME=${{ github.event.repository.name }}" >> $GITHUB_ENV
# Runs a set of commands using the runners shell
- name: Run a multi-line script
run: |
ls
sed -i 's/<reponame>/$REPO_NAME/' README.md
cat README.md
echo $REPO_NAME
I figured it out. You can it manually by using the sed command in bash within a runner in github actions. Set your README.md with a variable that you want to replace like <reponame> then use github actions to find that string and replace it with something you want (for me the repo name).
name: Dynamic Template
on:
create:
jobs:
update_templates:
name: "Update Templates"
runs-on: ubuntu-latest
steps:
- name: "📥 Fetching Repository Contents"
uses: actions/checkout#main
# Runs a set of commands using the runners shell
- name: Update README.md
run: |
sed -i 's/<reponame>/'${{ github.event.repository.name }}'/' README.md
git config user.email "41898282+github-actions[bot]#users.noreply.github.com"
git config user.name "github-actions[bot]"
git commit -am "Automated report"
git push
The email I used is a dependabot mentioned from here: https://github.com/orgs/community/discussions/26560#discussioncomment-3531273
This method needs four files
README.md
python script #replace tag
github_action.yml #run it automatically
variable.json #define variable
#PythonKiddieScripterX provides a good idea.
Based on the idea that context inside <> those <information will be hidden>. We can update our readme files by replacing those tags <> with the texts we want.
Example of my README.md file
This answer is version `1.0`. It will be updated automatically by loading `json` file and using GitHub Action.
In my readme.json file
{
VERSION: 1.0
}
Then I will only need a script to do the replacement. In this example, I use python saved as
replace_tag.py.
import re
import os
import json
# read all .md files
readmefiles = []
for root, dirs, files in os.walk("."):
for file in files:
if file.endswith(".md"):
readmefiles.append(os.path.join(root, file))
# load variable json
with open('readme.json') as f:
var_dic = json.load(f)
# match pattern (<variable-*.?-tag>)(`*.?`)
# example: (<variable-VERSION-tag>)(`1.1`)
for filename in readmefiles:
with open(filename,"r") as f:
content = f.read()
# update readme variables
for key, value in var_dic.items():
pattern = r"(<variable-{}-tag>)(`.*?`)".format(key)
replacement = r"\1`{}`".format(value)
content = re.sub(pattern, replacement, content)
with open(filename,"w") as f:
f.write(content)
Then the final thing is to use GitHub Action to run this python script every time there is a change.
yml file could be
name: README Dynamic Update
on:
push:
paths:
- *.md
- readme.json
- **/*.md
workflow_dispatch:
jobs:
update_templates:
name: "Update Templates"
runs-on: ubuntu-latest
steps:
- name: "📥 Update GitHub Readme files"
uses: actions/checkout#main
# Runs a set of commands using the runners shell
- name: Update README.md
run: |
python replace_tag.py
- name: pull-request
uses: repo-sync/pull-request#v2
with:
destination_branch: "main"
github_token: ${{ secrets.GITHUB_TOKEN }}
- name: commit
run: |
git config --global user.email youremail
git config --global user.name yourusername
git add .
git commit -m "README update Automation" -a
- name: Push changes
uses: ad-m/github-push-action#master
with:
github_token: ${{ secrets.GITHUB_TOKEN }}
Also for those text inside the code blocks, we can use <pre></pre> to solve the hidding issue.
Note that
need to add tag in this format <variable-YOURTAG-tag>"variable"

cmake does not (always) order Fortran modules correctly

I have a code using Fortran modules. I can build it with no problems under normal circumstances. CMake takes care of the ordering of the module files.
However, using a gitlab runner, it SOMETIMES happens that cmake does NOT order the Fortran modules by dependencies, but alphabetically instead, which than leads to a build failure.
The problem seems to occur at random. I have a branch that built in the CI. After adding a commit, that modified a utility script not involved in any way in the build, I ran into this problem. There is no difference in the output of the cmake configure step.
I use the matrix configuration for the CI to test different configurations. I found, that I could trigger this by adding another mpi version (e.g. openmpi/4.1.6). Without that version, it built. With it added in the matrix, ALL configurations showed the problem.
stages:
- configure
- build
- test
.basic_config:
tags:
- hpc_runner
variables:
# load submodules
GIT_SUBMODULE_STRATEGY: recursive
.config_matrix:
extends: .basic_config
# define job matrix
parallel:
matrix:
- COMPILER: [gcc/9.4.0]
PARALLELIZATION: [serial, openmpi/3.1.6]
TYPE: [option1, option2]
BUILD_TYPE: [debug, release]
- COMPILER: [gcc/10.3.0, intel/19.0.5]
PARALLELIZATION: [serial]
TYPE: [option2]
BUILD_TYPE: [debug]
###############################################################################
# setup script
# These commands will run before each job.
before_script:
- set -e
- uname -a
- |
if [[ "$(uname)" = "Linux" ]]; then
export THREADS=$(nproc --all)
elif [[ "$(uname)" = "Darwin" ]]; then
export THREADS=$(sysctl -n hw.ncpu)
else
echo "Unknown platform. Setting THREADS to 1."
export THREADS=1
fi
# load environment
- source scripts/build/load_environment $COMPILER $BUILD_TYPE $TYPE $PARALLELIZATION
# set path for build folder
- build_path=build/$COMPILER/$PARALLELIZATION/$TYPE/$BUILD_TYPE
configure:
stage: configure
extends: .config_matrix
script:
- mkdir -p $build_path
- cd $build_path
- $CMAKE_COMMAND
artifacts:
paths:
- build
expire_in: 1 days
###############################################################################
# build script
build:
stage: build
extends: .config_matrix
script:
- cd $build_path
- make
artifacts:
paths:
- build
expire_in: 1 days
needs:
- configure
###############################################################################
# test
test:
stage: test
extends: .config_matrix
script:
- cd $build_path
- ctest --output-on-failure
needs:
- build
The runner runs on an HPC machine which a complex setup, and I am not to familiar with the exact configuration. I contacted the admin with this problem, but wanted to see if anybody else had run into this before and have solutions or hints on what is going on.
With the help from our admin I figured it out.
The problem comes from cmake using absolute paths. The runner has actually several runners for parallel jobs, with each using a different prefix path, e.g. /runner/001/ or /runner/012/. So when I run configure on a specific runner, cmake saves that prefix path to the configuration.
Now in the build stage, there is no guarantee to have the same configuration run on the same runner. However, since there are absolute paths in the make files, make tries to access the folders in the configure runner's prefix. Now, that can be anything from non-existing, over old files from previous pipelines to the correct files downloaded by another case.
The only fix I currently can see is to run everything on the same runner in one stage, to avoid the roulette of prefix paths. If anybody has a different idea, or if there is a way to fix a specific matrix case to a specific runner prefix, please comment.

Why does my terminal give me an error when trying to cd to another directory

I am using iTerm and oh-my-zsh. When I try and cd into a directory on the command line I get this message Can't find a suitable configuration file in this directory or any parent: not found.
Here is an example: cd laracan't find a suitable configuration file in this directory or any parent: not found dock
.zshrc
# Path to your oh-my-zsh installation.
export ZSH="/Users/blake/.oh-my-zsh"
# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/robbyrussell/oh-my-zsh/wiki/Themes
ZSH_THEME="amuse"
# Which plugins would you like to load?
# Standard plugins can be found in ~/.oh-my-zsh/plugins/*
# Custom plugins may be added to ~/.oh-my-zsh/custom/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(git)
source $ZSH/oh-my-zsh.sh
# User configuration
# Completions Path
fpath=(/usr/local/share/zsh-completions $fpath)
## ALIASES
alias ls='ls -GFh'
alias ll='ls -al'
alias cpwd='pwd | pbcopy; pwd'
## GIT ALIASES
alias gs='git status'
# Checkout
alias co='git checkout'
alias cot='git checkout test'
# Pull and Push
alias pullo='git pull origin'
alias pusho='git push origin'
# Merge
alias gm='git merge'
alias mert='git merge test'

BitBucket deployment using SSH keys to remote server

I am trying to write a YAML pipeline script to deploy files that have been altered from my bitbucket repository to my remote server using ssh keys. The document that I have in place at the moment was copied from bitbucket itself and has errors:
pipelines:
default:
- step:
name: Deploy to test
deployment: test
script:
- pipe: atlassian/sftp-deploy:0.3.1
- variables:
USER: $USER
SERVER: $SERVER
REMOTE_PATH: $REMOTE_PATH
LOCAL_PATH: $LOCAL_PATH
I am getting the following error
Configuration error
There is an error in your bitbucket-pipelines.yml at [pipelines > default > 0 > step > script > 1]. To be precise: Missing or empty command string. Each item in this list should either be a single command string or a map defining a pipe invocation.
My ssh public and private keys are setup in bitbucket along with the fingerprint and host. The variables have also been setup.
How do I go about setting up my YAML deploy script to connect to my remote server via ssh and transfer the files?
Try to update the variables section become:
- variables:
- USER: $USER
- SERVER: $SERVER
- REMOTE_PATH: $REMOTE_PATH
- LOCAL_PATH: $LOCAL_PATH
Here is am example about how to set variables: https://confluence.atlassian.com/bitbucket/configure-bitbucket-pipelines-yml-792298910.html#Configurebitbucket-pipelines.yml-ci_variablesvariables
Your directive - step has to be intended.
I have bitbucket-pipelines.yml like that (using rsync instead of ssh):
# This is a sample build configuration for PHP.
# Check our guides at https://confluence.atlassian.com/x/e8YWN for more examples.
# Only use spaces to indent your .yml configuration.
# -----
# You can specify a custom docker image from Docker Hub as your build environment.
image: php:7.2.1-fpm
pipelines:
default:
- step:
script:
- apt-get update
- apt-get install zip -y
- apt-get install unzip -y
- apt-get install libgmp3-dev -y
- curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer
- composer install
- cp .env.example .env
#- vendor/bin/phpunit
- pipe: atlassian/rsync-deploy:0.2.0
variables:
USER: $DEPLOY_USER
SERVER: $DEPLOY_SERVER
REMOTE_PATH: $DEPLOY_PATH
LOCAL_PATH: '.'
I suggest to use their online editor in repository for editing bitbucket-pipelines.yml, it checks all formal yml structure and you can't commit invalid file.
Even if you check file on some other yaml editor, it may look fine, but not necessary according to bitbucket specification. Their online editor does fine job.
Also, I suggest to visit their community on atlasian community as it's very active, sometimes their staff members are providing answers.
However, I struggle with plenty dependencies needed to run tests properly. (actual bitbucket-pipelines.yml is becoming bigger and bigger).
Maybe there is some nicely prepared Docker image for this job.

gitlab-ci job not running script

I am new to gitlab-ci and trying a minimal Python application based on a gitlab template.
My .gitlab-ci.yml file is below:
# This file is a template, and might need editing before it works on your project.
# Official language image. Look for the different tagged releases at:
# https://hub.docker.com/r/library/python/tags/
#image: python:latest
# Change pip's cache directory to be inside the project directory since we can
# only cache local items.
variables:
PIP_CACHE_DIR: "$CI_PROJECT_DIR/.cache"
stages:
- test
- run
# Pip's cache doesn't store the python packages
# https://pip.pypa.io/en/stable/reference/pip_install/#caching
#
# If you want to also cache the installed packages, you have to install
# them in a virtualenv and cache it as well.
#cache:
# paths:
# - .cache/pip
# - venv/
before_script:
- python -V # Print out python version for debugging
#- pip install virtualenv
#- virtualenv venv
- python -m venv venv
- venv/scripts/activate
job1:
stage: test
script:
- python setup.py test
#- pip install tox flake8 # you can also use tox
#- tox -e py36,flake8
job2:
stage: run
script:
- pip install wheel
- python setup.py bdist_wheel
artifacts:
paths:
- dist/*.whl
#pages:
# script:
# - pip install sphinx sphinx-rtd-theme
# - cd doc ; make html
# - mv build/html/ ../public/
# artifacts:
# paths:
# - public
# only:
# - master
The jobs are seen within the gitlab web UI and they appear to run on my (windows based shell executor) runner.
When I look at the output for the jobs, it appears as if the actual script commands for each job aren't running at all.
Here's the output from job1:
Running with gitlab-runner 11.2.0 (35e8515d)
on GKUHN-L04 b0162458
Using Shell executor...
Running on GKUHN-L04...
Fetching changes...
Removing venv/
HEAD is now at 2484105 And agai..
From https://gitlab.analog.com/GKuhn/test_gitlab_ci
- [deleted] (none) -> origin/test_ci
fdd4216..cd618ba master -> origin/master
Checking out cd618ba9 as master...
Skipping Git submodules setup
$ python -V
Python 3.7.0
$ python -m venv venv
$ venv/scripts/activate
Job succeeded
And job2:
Running with gitlab-runner 11.2.0 (35e8515d)
on GKUHN-L04 b0162458
Using Shell executor...
Running on GKUHN-L04...
Fetching changes...
Removing venv/
HEAD is now at cd618ba updated .gitlab-ci.yml file
Checking out cd618ba9 as master...
Skipping Git submodules setup
$ python -V
Python 3.7.0
$ python -m venv venv
$ venv/scripts/activate
Uploading artifacts...
WARNING: dist/*.whl: no matching files
ERROR: No files to upload
Job succeeded
What am I doing wrong??
Turns out this is bug on Windows with the shell executor:
https://gitlab.com/gitlab-org/gitlab-runner/issues/2730
And a duplicate of this question: Gitlab CI does not execute npm scripts