We are trying to manage our own C++ static libraries using JFrog Artifactory CE. In the near future, these libraries could be accessed by third parties so we don't want to put any .cpp files in the package, we just want to put .h files and compiled libraries in our conan packages hosted on Artifcatory.
I read through the conan official guide https://docs.conan.io/en/1.3/creating_packages.html
https://docs.conan.io/en/1.3/creating_packages/package_repo.html
but I cannot find any description of how to exclude source files from the recipe.
If I don't specify exports_sources or exports in my conanfile.py I cannot build static libraries but if I specify those parameters, conan puts source files under export/conan_sources.tgz automatically when I execute conan create.
How can I create a conan package without including source files in the recipe?
There are 2 different ways to do this, instead of using the exports_... functionality:
Use the source() method, to retrieve whatever tarball, git-clone, or what is necessary to fetch the sources. This might require some authentication, which can be provided through env-vars. It is typical to use the conandata.yml to put the data there, and let the source() method to read the self.conan_data. Check this docs. The recipes in the conan-center-index repo, that serves to build ConanCenter, uses this approach.
Use the scm component if the recipe lives in the same repo as the source code, to capture the URL and commit of the sources, but without capturing the sources. If the scm code is behind auth, only authorized devs will be able to see the sources or build from sources. Check this section of the docs about SCM
In both cases, if the access to the source is restricted, non-privileged users that try to build packages from sources with --build will fail.
I am the author of KiKit. KiKit is a tool using KiCAD's Python API. KiKit also serves as a plugin for KiCAD. To register a plugin for KiCAD, you have to place a Python file in a specific location (e.g., /usr/share/kicad/scripting/plugins/ or ~/.kicad_plugins/ - see KiCAD doc for details).
I distribute KiKit as a python package via pip so my users can just type pip install kikit and not care about anything else. I would like to be able to register the plugin within the installation step so the users don't have to perform this step manually.
I know that wheel does not support arbitrary code execution thus no post-installation script is possible. But I wasn't able to find out if it is possible to include a file that will be installed on a location outside the root of the package. If so, how can I specify it in setup.py?
PS: I am aware of the option to provide an extra script, e.g., register-kikit, which would register the plugin. But I would like to include it in the installation step so there is a lesser chance of the user forgetting to do so.
The pipeline is triggered by new commits to a master branch and publishes the package
Currently, the version is set manually and I'de be happy to have it set automatically.
What I thought at first was adding the following tasks to the pipeline:
checkout $Build.SourceBranch
run version patch --force
git push
This works and the version is really incremented properly, the thing is, it triggers another run of the pipeline, which will increment again, which will.. you get the idea, endless loop.
Is there a better way for doing that?
I've added the same tasks as mentioned in the question with a small change.
Apparently there is a way to skip the pipeline triggering, see here
So the npm version task looks like this:
version patch -m "Bump version to %s [skip ci]" --force
which prevents the next build from being triggered.
TIP: remember to give the 'author' (Azure DevOps user) permissions to Bypass policies when pushing if any.
Don't use the files in the source repo to keep track of the current/next version. I don't think you can break the loop easily, if you do.
You might be able to get away with running npm --no-git-tag-version version to increment the package.json version inside the build agent without the commit, so that you don't have a change you would have to push back to the origin. It should just change package.json and leave it dirty.
Wait until after the build has succeeded. Use a custom script task to extract the version from package.json, then git reset --hard (there is no reason to keep anything that has changed on the build server). While this will undo the change in package.json, you can now create a tag on the head that contains that version, then do a git push origin {tag-name} which should not introduce a new commit on the origin which would then re-trigger your pipeline.
Actually, I don't think a pipeline will be triggered just because a tag was added, but I honestly haven't tested it. I'm pretty sure it won't.
Sequence of tasks in your pipeline:
[Given a source repo where you have made sure to set the major and minor versions in package.json to reflect the current state of the code according to the rules of Semantic Versioning, and leave the patch value always set to 0, and, if desired, using pre-* values to describe the quality of the major/minor value:]
(This is not really a task, but just describing the start of the execution:) The job starts. Automatically, the code is pulled from the source repo into the build agent.
Using a utility command-line task and code or script that you write, run git describe --tags to find the most recent tag with a tagname matching the pattern you use (see below). (I prefer this sequence over calling npm version from-git because npm will just use the latest tag, which might not be a version number, depending on how much control you have over the branch.) Use string or regular expression operations to extract the major, minor, patch, and whatever pre-* value you might have. This is the previous version that we'll use to compare with what's in the package.json file. Note that you may have to follow these instructions to run a git command. Save it to a pipeline variable.
Using a utility command-line task and code or script that you write, run npm version to get the current major/minor/pre version out of the package.json file and save it to a different pipeline variable. I'm using PowerShell Core, so my command would look something like this to create a "currentPackageVersion" pipeline variable:
& npm.cmd version | ConvertFrom-Json | Select-Object -ExpandProperty {name-of-your-package} | Set-Variable -Name 'packageVersion' | Write-Output "#vso[task.setvariable variable=currentPackageVersion]$packageVersion"
Using a utility command-line task and code or script that you write, compare the previous version's major, minor, and pre-* values to determine whether any of them have changed. Set a new pipeline variable to reflect whether it has changed or not. I'll use the name "restartVersionPatchNumber", which is true if the current major, minor, or pre-* values are different from the previous version's major, minor, or pre-* values.
The npm Task conditionally runs a custom command: npm --no-git-tag-version version patch, which updates package.json in the build agent but does not commit the change, leaving your Working Area modified (dirty) (which might cause issues on subsequent builds if you are using your own build agents instead of hosted agents). The condition expression of the Task uses a custom condition that evaluates to the variable that I just set in the previous step ("restartVersionPatchNumber"). Note that if this task does not run, it should just use the value of the version that is in the package.json file (the current version that is now in the "currentPackageVersion" pipeline variable).
Using a utility command-line task and code or script that you write, run npm version to extract the new version that the npm version command set. Save it to a new pipeline variable; I'll call it "newVersionNumber".
The regular build tasks run, producing artifacts and possibly publishing them
Using a utility command-line task, run git reset --hard. You'll need to do this even if you're using a hosted build agent, because of the next step.
Using a utility command-line task, create a variable from the saved version number ("newVersionNumber") that contains the value of the tag's tagname that you want to use. Use a distinctive patter, like "AzPipelineBuild-PipelineName-PackageVersion-1.0.0" but using your version instead of 1.0.0.
Using a utility command-line task, run git tag {*tagname*}. For PowerShell, the syntax would be & git.exe tag $env:newVersionNumber
Using a utility command-line task, run git push origin {*tagname*}
Profit
You could (and maybe should) combine the command-line steps. I'm really partial to the PowerShell task running PowerShellCore (pwsh) as you may have guessed.
The package that got created as an artifact of the job will have the updated version as it was created in the build, and it will match the tag that is now in your original source code repo.
Alternately, use an external source (an Azure Function, etc.) or a different (second) git repo or even another branch that isn't tied to your CI trigger in your project that you use just for tracking build numbers, then use token replacement tasks to set the version just before you start the build. I don't like this idea much, but it would prevent re-triggering a new build.
Also, you're hopefully building just one package with that repo. The next thing to do is to publish that package to your Azure Artifacts feed, or npmjs.org or wherever. Don't rely on having the version baked into the original source code; anything that depends on that package should be pulling it out of that feed you published it to, not relying on it being built earlier in the build steps with a new version number.
Here's my implementation in Powershell which is a complete step based on the accepted answer. Note that this also handles some of the git issues such as the detached head that comes with this type of approach.
steps:
- checkout: self
persistCredentials: true
.
.
.
- task: PowerShell#2
displayName: 'Bump the version'
inputs:
targetType: 'inline'
script: |
$BranchName = "$(Build.SourceBranch)" -replace "refs/heads/"
git checkout $BranchName
git config --global user.email "anymail#anycompany.com"
git config --global user.name "Your name"
npm version prerelease -m "Auto increment pre-release version to %s [skip ci]" --force
git push
We want to generate a new project to test few things on specific release for e.g; 16.11.2 (Kiwi release)
What -additional params that is required to pass with the given command
mvn -U org.seedstack:seedstack-maven-plugin:generate
Regards
Indrajit
You can generate a project for a specific version by manually specifying the archetype version (which always matches the SeedStack version).
For SeedStack 16.11.2:
mvn -U org.seedstack:seedstack-maven-plugin:generate -DarchetypeVersion=16.11.2
See the related documentation for additional parameters and information.
I'm using PMEase QuickBuild to perform automated builds of our Maven2 projects and a nightly sanity test to ensure nothing is broken.
The test needs to untar packages which are created by the automated Maven2 projects. The problem is that the package names change frequently due to project versions being incremented all the time.
Does anyone know how I can configure QuickBuild to pick up the version (ideally from the POM file of the individual components), if this is possible at all?
I don't know if this is an option for you but it looks like you can do it the other way around. Quoting Build with Maven:
Control build version
If you want to control the build
version from QuickBuild side, please
follow below steps:
Change the POM file and define the project version as
${buildVersion}. Do not forget to
commit the file into your SCM after
change.
Define a build property like below when define the Maven build
step:
buildVersion=${build.version}
There are maybe other options but I must admit that my knowledge (zero) of QuickBuild is very limited
I created a work around to this issue by having QuickBuild execute a shell script which did the untarring by using wildcards, similar to the following (to avoid computing the exact version):
tar xzf filename-*.tar.gz
I couldn't figure out how to do this in QuickBuild, so I offloaded the work to the shell script.