Tangentially related to: AWS CodeBuild with GitHub - only for specific directory
I have a codebuild project and a github repo with many files in it. A user may update any of these files in the git repo. I want to pass the name of the altered file(s) into my buildspec.yaml somehow; IE my merge job logic, specified in the buildspec.yaml, needs to know what files changed to do a per-file operation.
I am not talking about filters; Ie "only trigger this if X,Y,Z changed". Becuase the filter is there for a large number of XYZ, but I need to know which file(s) changed in my buildspec. IE something like $CHANGED_FILE_LIST.
I don't see this here: https://docs.aws.amazon.com/codebuild/latest/userguide/build-env-ref-env-vars.html
Maybe we have to do something like this: how to find out list of all changed files in git for full jenkins build and not for a particular commit?
git diff --name-only $GIT_PREVIOUS_COMMIT $GIT_COMMIT
but one would think this meta info could be provided by codebuild
I don't know if there's a blessed CodeBuild way to do this, but assuming you have access to a GitHub token in the build, you can query the GitHub metadata endpoint to get the info you need. Something like
curl -H "Authorization: token ${yourtoken}" -H "Accept: application/vnd.github.v3+json" https://api.github.com/repos/OWNER/REPO/commits/${CODEBUILD_SOURCE_VERSION} | jq '.files[].filename'
will return a list of files in the commit.
See https://docs.github.com/en/rest/commits/commits#get-a-commit for more details.
git diff --name-status $CODEBUILD_WEBHOOK_BASE_REF should do the trick
You can use git diff --name-only $$CODEBUILD_RESOLVED_SOURCE_VERSION $$CODEBUILD_WEBHOOK_PREV_COMMIT
Where $CODEBUILD_WEBHOOK_PREV_COMMIT is the commit id of the previous commit. And $CODEBUILD_RESOLVED_SOURCE_VERSION is the commit id of the actual one.
Inside a build phase you can check the change with:
-
|
if [ "$(git diff --name-only $CODEBUILD_WEBHOOK_PREV_COMMIT $CODEBUILD_RESOLVED_SOURCE_VERSION | grep -e <filde_path>)" != "" ]; then
#your code;
fi
Related
I follow instruction from this link to create an image container in Azure Container registry whenever a commit is done.
Everything is working fine and I appreciate we could retrieve the Run.ID
az acr task create -t acb:{{.Run.ID}} -n acb-win -r MyRegistry \
-c https://github.com/Azure/acr-builder.git -f Windows.Dockerfile \
--commit-trigger-enabled false --platform Windows/amd64
I see also that we can use another tag like {{.Run.Registry}} instead of {{.Run.ID}}.
I am curious to know which other tag exists. In my workflow i wonder if it possible to retrieve the commitID.
Is there anyone who succeed to retrieve the commitID ? I tester several combinaison but no luck.
Many thanks to the community.
I answer to myself and found this link https://github.com/MicrosoftDocs/azure-docs/blob/main/articles/container-registry/container-registry-tasks-reference-yaml.md#task-step-properties which explain clairly all variable can be used.
I'm trying to sort the list of artifacts from jfrog artifactory but getting (The requested URL returned error: 400 Bad Request), in the jfrog documentation (https://www.jfrog.com/confluence/display/JFROG/Artifactory+Comparison+Matrix) says it won't work for open source services. After we get list of artifacts need to delete old artifacts from subfolder in the artifactory repo. Tried with CLI and AQL but nothing worked.
Our repo url looks like this
http://domainname/artifactory/repo/folder/subfolder/test1.zip
Like test 1.zip we have many artifacts(let's say 50)in that subfolder. Looking for help on this, anyone pls me on this issue. Thanks.
While sorting is not supported in OSS versions, if you would like to delete artifacts older than a certain time period, you can use Relative Time Operators, parse the output, and use a script to delete those artifacts.
You can also specify a specific date. There are several Comparison Operators that you can use.
You can use the below AQL for reference:
curl -uadmin:password -XPOST "http://localhost:8082/artifactory/api/search/aql" -d 'items.find({"repo": "repo"}, {"path": "folder/subfolder"}, {"created" : {"$before" : "2minutes"}})' -H "Content-Type: text/plain"
I have 3 steps in my build. If third step failed, I need to run specific step, which have to run ONLY after fail of previous one. How to do it?
It doesn't look like there's an execution policy to "run on previous step failure". You could work around it by using the execution policy "even if some of previous steps failed" and having your particular build step handle the logic.
i.e. If previous step successful, do nothing. If previous step unsuccessful, do the failure case.
If you need to detect if previous step was a failure or not, you could consider making a part of your build creating a status file (if in_error file exists then perform your task)
I was surprised that TeamCity does not support it out of the box in 2021. But API gives you a lot usefull features and you can do it
As a solution you need to write bash script and call TeamCity API inside
setup API key in MySettings & Tools => Access token
create env variable with API token
create a step in your configuration with Execute step: Even if some of the previous steps failed
build own container with jq or use any existing container with jq support
place this bash script
#!/bin/bash
set -e -x
declare api_response=$(curl -v -H "Authorization: Bearer %env.teamcity_internal_api_key%" -H "Accept: application/json" %teamcity.serverUrl%/app/rest/latest/builds?locator=buildType:%system.teamcity.buildType.id%,running:any,canceled:all,count:2\&fields=build\(id,status\))
declare current_status=`echo ${api_response} | jq '.build[0].status'`
declare prev_status=`echo ${api_response} | jq '.build[1].status'`
if [ "$current_status" != "$prev_status" ]; then
do you code here
fi
some explanation of code above. with API call you get 2 last builds of current buildType. This is last build and previous build. After you assign variable with statuses and compare them in if statement. If you need to run some code in case of current build failed use
if [ "$current_status" = "FAILURE" ]; then
write your code here
fi
I'm cloning an SVN repository to git as part of our migration plan. I've hit various snags along the way, forcing me to continue the clone with a git svn fetch command. The most recent failure I can't figure out how to solve:
$ git svn fetch
Checksum mismatch: dc/trunk-4632-jh/dc-smtpd/lib/Qpsmtpd/Address.pm.t 8ce3aea3f47dc115e8fe53bd62d0f074cfe93ec6
expected: 59de969022e46135fa6dc7599fc2f3b4
got: 4334926a01c905cdb7fce71265e370c1
I found this related answer, however that solution doesn't work because git svn log is not yet functional, as the repo is not fully in place:
$ git svn log dc/trunk-4632-jh/dc-smtpd/lib/Qpsmtpd/Address.pm.t
fatal: ambiguous argument 'HEAD': unknown revision or path not in the working tree.
Use '--' to separate paths from revisions
log --no-color --first-parent --pretty=medium HEAD: command returned error: 128
How can I proceed?
Another answer to an old question but straight forward solutions are tough to find for this problem so hopefully this helps others.
I think this issue occurs due to a corrupted file during transfer. Not sure how or why it happens, but in my case, I get the same error at different revisions every time I do a new clone and sometimes not at all.
Using the questioners error message
$ git svn fetch
Checksum mismatch: dc/trunk-4632-jh/dc-smtpd/lib/Qpsmtpd/Address.pm.t
8ce3aea3f47dc115e8fe53bd62d0f074cfe93ec6
expected: 59de969022e46135fa6dc7599fc2f3b4
got: 4334926a01c905cdb7fce71265e370c1
The following steps allowed me to resume and progress :-
View all branches. These will all be remote branches. git branch -a
Checkout branch affected. git checkout remotes/origin/trunk-4632-jh
This will take some time to complete.
Find the last revision that the problematic file was changed. git svn log dc-smtpd/lib/Qpsmtpd/Address.pm.t
Note the highest revision #
Reset back to this rev. git svn reset -r (rev #) -p
Carry on. git svn fetch
Good luck.
I know this is old but maybe it will be helpful for future reference as all search results on this are not helpful.
I've hit similar issue on our huge repository which takes days to clone and unfortunately at one point I had to restart my machine. I am currently working out how to resolve the problem, so please keep in mind this is more a suggestion than tested solution.
I think you need to try creating a branch and checking out the commits you currently have from previous fetch:
git checkout -b master git-svn
After that is done you should have working tree up to that commit. Another fetches will probably fail due to object mismatch but at that point at least it should be possible to use "git svn reset" to revert faulty svn fetches (see OP's related answer link). If that's true find offending commit, reset before it and then continue fetching.
You might want to rebase and revert to state before that broken commit on your master branch or convert back to bare repository, if that's what you're after (in my case it is).
Hope this works. I'll post an update when my checkout is done (will take at least few hours... sigh).
Edit: That seemed to work. I successfully discarded some git-svn commits and am able to re-fetch them again. :)
Edit2: Make sure to reset until you don't get any object mismatch warnings on git svn fetch (otherwise you will run into the same issue soon).
Cheers,
Henryk
See also: Git svn rebase : checksum mismatch
In our case the additional treatment of the files (server-side includes in Apache) caused the checksum problem.
Disabling SSI in Apache's /etc/httpd.conf file for the period of migration by commenting out the
AddType text/html .shtml
AddOutputFilter INCLUDES .shtml
directives solved the problem, caused by the interpretation of .shtml files by the front-end Apache server, which produced a new content (and thus a new hash), other than the hash of the original file itself.
That means some files in the repository got corrupted. It can be caused by various reasons such as software bugs, bit rots in drives, etc. I was recently transitioning very old ~10GB svn repository to git, therefore some corruption was expected.
To fix the corruption, you basically need to dump the entire repository and import it while filtering the errors out. Note that our goal is to complete the import process no matter why or how the repository got corrupted. You cannot simply fix the corruption without having a backup and diffing through the revision files.
First basic one-off command you could use is:
svnadmin create repo2
svnadmin dump repo | sed '/^Text-content-md5/d' | svnadmin load repo2
This removes the checksum calculation from the dump so the new repo will have updated checksums.
If you encountered more errors during the dump and load (which is expected), try incremental approach so you can continue from the point you left. Below command will dump the revisions starting from 101 to 150 (inclusive).
svnadmin dump --incremental -r101:150 repo | sed '/^Text-content-md5/d' | svnadmin load repo2
Some common errors and solutions:
'Premature end of content data in dumpstream': That means Content-length of some file does not match the repository version, so some data is lost in the specified file. We must skip it. Add | svndumpfilter exclude path/to/file.jar command like this:
svnadmin dump --incremental -r101:150 repo | svndumpfilter exclude path/to/file.jar | sed '/^Text-content-md5/d' | svnadmin load repo2
Property errors: Add --bypass-prop-validation to svnadmin load command
After populating your second repo, you would simply svnserve -d -r repo2 and try git svn fetch again.
Good luck!
I found a lot of examples on how to change the username for specific revisions and so on.
But what I need is this:
I did a checkout with the authentication credentials of a workmate and need to change it to my credentials for future commits.
I cannot just checkout with my credentials due to the many changes that have been done already...
Anyone familiar with this?
You can change the user with
Subversion 1.6 and earlier:
svn switch --relocate protocol://currentUser#server/path protocol://newUser#server/path
Subversion 1.7 and later:
svn relocate protocol://currentUser#server/path protocol://newUser#server/path
To find out what protocol://currentUser#server/path is, run
svn info
in your working copy.
The easiest way to do this is to simply use the --username option on your next checkout or commit. For example:
svn commit --username newUser
or
svn co --username newUser
It will then be cached and will be used as the default username for future commands.
See also:
In Subversion can I be a user other than my login name?
I’ve had the exact same problem and found the solution in Where does SVN client store user authentication data?:
cd to ~/.subversion/auth/.
Do fgrep -l <yourworkmatesusernameORtheserverurl> */*.
Delete the file found.
The next operation on the repository will ask you again for username/password information.
(For Windows, the steps are analogous; the auth directory is in %APPDATA%\Subversion\).
Note that this will only work for SVN access schemes where the user name is part of the server login so it’s no use for repositories accessed using file://.
The command, that can be executed:
svn up --username newUsername
Works perfectly ;)
P.S. Just a hint: "--username" option can be executed on any "svn" command, not just update.
Go to Tortoise SVN --> Settings --> Saved Data.
There is an option to clear Authentication Data, click on the clear button, and it will allow you to select which connection you wanted to clear userid/pwd for.
After you do this, any checkout or update activity, it will reprompt for the userid and password.
If your protocol is http and you are using Subversion 1.7, you can switch the user at anytime by simply using the global --username option on any command.
When Ingo's method didn't work for me, this was what I found that worked.
for Win10 you should remove this folder and close/open your IDE
C:\Users\User\AppData\Roaming\Subversion\auth
, also in my projects no ".subversion" folders, only ".svn"
Also, for those who happened to realize too late, that they committed with the wrong credentials, the solution (after the fact) is to change the svn author of that revision: see this question
Basically the syntax is:
svn propset --revprop -r (revision_number) svn:author (new username)
Based on Ingo Kegel's solution I created a "small" bash script to change the username in all subfolders. Remember to:
Change <NEW_USERNAME> to the new username.
Change <OLD_USERNAME> to the current username (if you currently have no username set, simply remove <OLD_USERNAME>#).
In the code below the svn command is only printed out (not executed). To have the svn command executed, simply remove the echo and whitespace in front of it (just above popd).
for d in */ ; \
do echo $d ; pushd $d ; \
url=$(svn info | grep "URL: svn") ; \
url=$(echo ${url#"URL: "}) ; \
newurl=$(echo $url | sed "s/svn+ssh:\/\/<OLD_USERNAME>#/svn+ssh:\/\/<NEW_USERNAME>#/") ; \
echo "Old url: "$url ; echo "New url: "$newurl ; \
echo svn relocate $url $newurl ; \
popd ; \
done
Hope you find it useful!
I believe you could create you own branch (using your own credential) from the same trunk as your workmate's branch, merge from your workmate's branch to your working copy and then merge from your branch. All future commit should be marked as coming from you.
You could ask your colleague to create a patch, which will collapse all the changes that have been made into a single file that you can apply to your own check out. This will update all of your files appropriately and then you can revert the changes on his side and check yours in.
Tortoise on windows specific:
I attempted the above solution to no available. In my use case, the credentials where stored in the windows Credential Manager under the Windows Credential section and TortoiseSVN refused to change them by running the --username command.
You can edit the credentials by expanding the url for your repo and clicking edit.