I'm creating a pipeline for manage Wordpress themes building and I need to make sure that when a user merge the code on the master branch the theme version is different from the one already present in master.
For example I have in master the 1.0.0 version, I want to make sure that after the merge the version is greather (or at least different ) from 1.0.0.
This is my stage:
check_theme_version_number:
stage: checks
script:
- git fetch
- lines=$(git diff origin/master origin/$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME -- wp-content/themes/$THEME_NAME/style.css | grep Version | wc -l)
- if (($lines == 0)); then exit 1; fi
only:
- master
$THEME_NAME is passed from Gitlab variables.
When I run this stage I get:
$ git fetch
From https://gitlab.com/xxxxx/xxxxx/xxxxx
* [new branch] develop -> origin/develop
$ lines=$(git diff origin/master origin/$CI_MERGE_REQUEST_SOURCE_BRANCH_NAME -- wp-content/themes/$THEME_NAME/style.css | grep Version | wc -l)
fatal: bad revision 'origin/'
Cleaning up project directory and file based variables
00:01
ERROR: Job failed: exit code 1
Related
I really dont know if this is a simple (must be), common or complex task.
I have a buildspec.yml file in my codebuild project, and i am trying to append the version written in package.json file to the output artifact.
I have already seen a lot of tutorials that teach how to append the date (not really useful to me), and others that tell me to execute a version.sh file with this
echo $(sed -nr 's/^\s*"version": "([0-9]{1,}.[0-9]{1,}.*)",$/\1/p' package.json)
and set it in a variable (it doesn't work).
i'm ending up with a build folder called: "my-project-$(version.sh)"
codebuild environment uses ubuntu and nodejs
Update (solved):
my version.sh file:
#!/usr/bin/env bash
echo $(sed -nr 's/^\s*\"version": "([0-9]{1,}\.[0-9]{1,}.*)",$/\1/p' package.json)
Then, i just found out 2 things:
Allow access to your version.sh file:
git update-index --add --chmod=+x version.sh
Declare a variable in any phase in buildspec, i dit in in build phase (just to make sure repository is already copied in environment)
TAGG=$($CODEBUILD_SRC_DIR/version.sh)
then reference it in artifact versioned name:
artifacts:
files:
- '**/*'
name: workover-frontend-$TAG
As result, my build artifact's name: myproject-1.0.0
In my case this script do not want to fetch data from package.json. On my local machine it working great but on AWS doesn't. I had to use chmod in different way, because i got message that i don't have right permissions. My buildspec:
version: 0.2
env:
variables:
latestTag: ""
phases:
pre_build:
commands:
- "echo sed version"
- sed --version
build:
commands:
- chmod +x version.sh
- latestTag=$($CODEBUILD_SRC_DIR/version.sh)
- "echo $latestTag"
artifacts:
files:
- '**/*'
discard-paths: yes
And results in console:
CodeBuild
I also have to mark that when i paste only for example echo 222 into version.sh file i got right answer in CodeBuild console.
I've searched a lot of examples but they did not work for me.
I'm trying to run linters for changed files when MR is opened.
My .gitlab-ci.yml
run_linters:
image: python:3
variables:
FILES: git diff --name-only $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep *.py
before_script:
- python3 -m pip install black==21.5b1
- python3 -m pip install flake8==3.9.2
script:
- echo $FILES
- git diff --name-only $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep *.py
- black --check $FILES
- flake8 $FILES
only:
- merge_requests
And I'm getting strange output.
echo $FILES says git diff --name-only main | grep incoming_file.py
incoming_file.py is the only file in that MR. Why is it around grep?
And git diff at script section says fatal: ambiguous argument 'main': unknown revision or path not in the working tree.
Why is filename present around grep?
Why are same git diff commands give different result?
Why is filename present around grep?
In bash when you refer to * this will expand and try to match the files/directories present in your current path, in your case since only the incoming_file.py is present, so it expands to this.
Why are same git diff commands give different result?
variables:
FILES: git diff --name-only $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep *.py
When you define a variable in variables section, Gitlab doesnt execute the command, it simple populates the variable FILES with the string git diff --name-only $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep *.py
Then in the script section, the runner expands *.py to incoming_file.py and $CI_MERGE_REQUEST_TARGET_BRANCH_NAME to main
that's why in echo you see git diff --name-only main | grep incoming_file.py
Here
- git diff --name-only $CI_MERGE_REQUEST_TARGET_BRANCH_NAME | grep *.py
You actually execute the command and you get the mentioned message
I am new to gitlab-ci and I am trying to test my build phase but it keeps failing before last step. here is the section
build:
stage: build
before_script:
- git config user.name "$GITLAB_USER_NAME"
- git config user.email "$GITLAB_USER_EMAIL"
script:
- git add .
- git commit -m "[skip ci] migrations"
- git push "https://${GITLAB_USER_NAME}:${CI_GIT_TOKEN}#${CI_REPOSITORY_URL}" "HEAD:qa" -o ci.skip
dependencies:
- migrations
only:
- qa
This is the log, last few lines
Using docker image sha256:5b3b4504ff1f7b859dbc5d7fb86f4afc644be62f99b8ced636fbca64c8a6c2de for python:latest with digest python#sha256:73cc381fa0fe5e6d5dd38f1397da28c70096519d8818c2249f2bc8e1158ba015 ...
$ git config user.name "$GITLAB_USER_NAME"
$ git config user.email "$GITLAB_USER_EMAIL"
$ git add .
$ git commit -m "[skip ci] migrations"
HEAD detached at bf2a8e4
nothing to commit, working tree clean
Cleaning up file based variables
00:01
ERROR: Job failed: exit code 1
How can I find out what ERROR: Job failed: exit code 1 is? I am using shared runners
The error ERROR: Job failed: exit code 1 is coming from git not finding anything to commit. In other words, the below message is causing the error:
nothing to commit, working tree clean
Cleaning up file based variables
You can either may sure there is something to commit, or you can adjust your script as outlined in this related thread: How to git commit nothing without an error?
I'm trying to migrate an 11GB SVN repo with over than 24k revisions inside to a single GIT repository.
I did a single file dump of the SVN using svnrdump command and load it into my local SVN server, placed on my MacBook machine.
I downloaded the svn2git from the https://github.com/svn-all-fast-export/svn2git repository.
Due to differences in the way how SVN and GIT handle tags, I used a merged-branches-tags.rules from the svn2git sample directory, which look like this (I've removed comments):
create repository myproject
end repository
match /trunk/
repository myproject
branch master
end match
match /(branches|tags)/([^/]+)/
repository myproject
branch \2
end match
Then I used a docker image solution as described in the documentation (in my console it was a single line. I've did split it to clarify what I was doing):
docker run --rm -it \
-v /Users/me/work/SVN/dest:/workdir \
-v /Users/me/work/svnServer/repositories/my_svn_repo:/tmp/svn \
-v /Users/me/work/SVN/svn2git/samples:/tmp/conf \
svn2git /usr/local/svn2git/svn-all-fast-export \
--rules /tmp/conf/merged-branches-tags.rules \
--add-metadata --svn-branches --debug-rules --svn-ignore --empty-dirs \
/tmp/svn/
During the first try I got an error between revisions 12600 and 126001:
Exporting revision 12601 /tags/7.0M0p0000 was copied from /tags rev 12600
rev 12601 /tags/7.0M0p0000/ matched rule: "/tmp/conf/merged-branches-tags.rules:28 /(branches|tags)/([^/]+++++
)/" exporting.
.WARN: SVN reports a "copy from" # 12601 from /tags # 12600 but no matching rules found! Ignoring copy, treating as a modification
WARN: Transaction: "7.0M0p0000" is not a known branch in repository "myproject"
Going to create it automatically
add/change dir ( /tags/7.0M0p0000 -> "7.0M0p0000" "" )
+++++
WARN: Branch "7.0M0p0000" in repository "myproject" doesn't exist at revision 12601 -- did you resume from the wrong revision?
Failed to write to process: Process crashed for repository myproject
6223345 modifications from SVN /tags/7.0M0p0000/ to myproject/7.0M0p0000%
I've check it and in the rev 12601 there there is a new tag named as "7.0M0p0000", which I'm going to import as a branch and which wasn't in the repo in rev 12600.
Do you have any ideas what can I do to fix that and continue my migration?
Any help will be really appreciated.
After a further investigation, it turns out that the mentioned "7.0M0p0000" tag was created in the rev 12601 as a copy of all the tags from rev 12600.
I've found it in the dump file, created using this command:
svnrdump dump -r 12600:12601 --incremental http://xxx.xxx.xxx.xxx/svn/my_repo > my_repo.dump
There was an entry:
Revision-number: 12601
...
Node-path: tags/7.0M0p0000
Node-kind: dir
Node-action: add
Node-copyfrom-rev: 12600
Node-copyfrom-path: tags
It seems that KDE's svn2git is unable to deal with such cases (which was probably done by mistake).
The only solution I found was completely skip this tag by adding a match to my merged-branches-tags.rules file (order of match expressions is important):
match /tags/7.0M0p0000/
min revision 12600
max revision 12606
end match
...
match /(branches|tags)/([^/]+)/
repository myproject
branch \2
end match
I've been working on setting up an automated RPM build and I'd like to perform a simple test on the SPEC file before proceeding with any build steps. The problem I am having is that the job always seems to jump to the deploy stage. Here is the relevant snippet from my .gitlab-ci.yml:
stages:
- test
- build
- deploy
job1:
stage: test
script:
# Test the SPEC file
- su - newbuild -c "rpmbuild --nobuild -vv ~/rpmbuild/SPECS/package.SPEC"
stage: build
script:
# Install our required packages
- yum -y install openssl-devel freetype-devel fontconfig-devel libicu-devel sqlite-devel libpng-devel libjpeg-devel ruby
# Initialize the submodules to build
- git submodule update --init
# build the RPM
- su - newbuild -c "rpmbuild -ba --target=`uname -m` -vv ~/rpmbuild/SPECS/package.SPEC"
stage: deploy
script:
# move the RPM/SRPM
- mkdir -pv $BUILD_DIR/$RELEASEVER/{SRPMS,x86_64}
- 'for f in $WORK_DIR/rpmbuild/RPMS/x86_64/*; do cp -v "$f" $BUILD_DIR/$RELEASEVER/x86_64; done'
- 'for f in $WORK_DIR/rpmbuild/SRPMS/*; do cp -v "$f" $BUILD_DIR/$RELEASEVER/SRPMS; done'
# create the repo
- createrepo -dvp $BUILD_DIR/$RELEASEVER
# update latest
- 'if [ $CI_BUILD_REF_NAME == "master" ]; then rm $PROJECT_DIR/latest; ln -sv $(basename $BUILD_DIR) $PROJECT_DIR/latest; fi'
- 'if [ $CI_BUILD_REF_NAME == "devel" ]; then rm $PROJECT_DIR/latest-dev; ln -sv $(basename $BUILD_DIR) $PROJECT_DIR/latest-dev; fi'
tags:
- repos
I've not found any questions or online documentation to properly explain this to me so any help is appreciated!
You have all stages in one job which does not work. You need to split it up into individual jobs for the three different stages.
Quote from the documentation:
First all jobs of build are executed in parallel.
If all jobs of build succeeds, the test jobs are executed in parallel.
If all jobs of test succeeds, the deploy jobs are executed in parallel.
If all jobs of deploy succeeds, the commit is marked as success.
If any of the previous jobs fails, the commit is marked as failed and no jobs of further stage are executed.
Something like this should work:
stages:
- test
- build
- deploy
do_things_on_stage_test:
script:
- do things
stage: test
do_things_on_stage_build:
script:
- do things
stage: build
do_things_on_stage_deploy:
script:
- do things
stage: deploy
I think you assume that the stages are build on top of each other, which is not the case. If one of your stages needs something like pre-installed packages, you have to add a before_script directive. Think of the stages as in: test-if-build-succeeds, test-if-depoy-succeeds, etc.