Considering using Conan.io to manage the enormous amount of binaries created by our CI system.
Currently, source for several (~50) SW modules is versioned in SVN and we use SVN externals to specify which modules should be checked-out for each project.
So far I understand that Conan uses so-called "recipes" to define the needed sources. Can Conan understand SVN externals and record their location, and their revision, as to how the package was built? I can't find any information on the doc website.
Any example on how to define svn repos with externals in the Conan recipe would be very helpful!
Thanks
Related
I want to have a rpm package build with Copr1. My current build target list is Fedora 35, 36, rawhide and Centos 7 and Stream 8. I have not yet created the copr project.
Compiling on one of my machines, the package builds successfully on the Fedora variants with mock. The problem is that on Centos variants one of the build dependencies and some of its dependencies are not available. I have found appropriate srpm files and compiled them on one of my machines with the Centos Stream 8 (one of them required two custom patches). With those custom dependencies I am able to successfully compile the original package.
So just to be clear, the problem is that the spec file contains for example
BuildRequires: libsomething
where libsomething is available as a plain upstream package in some of the build targets while needs an additional custom repo for some other build targets.
The FAQ says the following about dependencies:
Can I depend on other packages, which are not in Fedora/EPEL?
Yes, they just need to be available in some yum repository. It can either be another Copr repo or a third-party yum repo (e.g jpackage). Click on “Edit” in your project and add the appropriate repositories into the “Repos” field. Packages from your project are available to be used at build time as well, but only for the project you are currently building and not from your other projects.
But this sounds like an all or nothing approach, and I absolutely do not want to override the already existing upstream packages, only provide them when they are missing.
So what strategy do people use to handle this?
Update: I have now created copr projects and made some attempts at building (after resolving dependencies of dependencies in several levels), but the problem is as I describe above. If I add copr://hlovdal/projectname as a build dependency then epel-8-x86_64 compiles fine because it is provided with the missing dependencies while fedora-35-x86_64 fails because the repository does not have any fedora packages. If I remove the repo epel fails while fedora succeeds.
I also attempted to add the base url from the corresponding /etc/yum.d.repo file, and only hardcode epel instead of $distname hoping that the fedora builds would just ignore non-existing/wrong repo setting, but the build does not like that and still fails.
1 Copr is Fedora's freely available build system.
We are trying to manage our own C++ static libraries using JFrog Artifactory CE. In the near future, these libraries could be accessed by third parties so we don't want to put any .cpp files in the package, we just want to put .h files and compiled libraries in our conan packages hosted on Artifcatory.
I read through the conan official guide https://docs.conan.io/en/1.3/creating_packages.html
https://docs.conan.io/en/1.3/creating_packages/package_repo.html
but I cannot find any description of how to exclude source files from the recipe.
If I don't specify exports_sources or exports in my conanfile.py I cannot build static libraries but if I specify those parameters, conan puts source files under export/conan_sources.tgz automatically when I execute conan create.
How can I create a conan package without including source files in the recipe?
There are 2 different ways to do this, instead of using the exports_... functionality:
Use the source() method, to retrieve whatever tarball, git-clone, or what is necessary to fetch the sources. This might require some authentication, which can be provided through env-vars. It is typical to use the conandata.yml to put the data there, and let the source() method to read the self.conan_data. Check this docs. The recipes in the conan-center-index repo, that serves to build ConanCenter, uses this approach.
Use the scm component if the recipe lives in the same repo as the source code, to capture the URL and commit of the sources, but without capturing the sources. If the scm code is behind auth, only authorized devs will be able to see the sources or build from sources. Check this section of the docs about SCM
In both cases, if the access to the source is restricted, non-privileged users that try to build packages from sources with --build will fail.
I am distributing a Java package via git for other people to use. I am currently supplying a jar file to go with the source. This way, the user only needs to clone the project once into Intellij IDEA. Projects using the package can then follow this procedure
Correct way to add external jars (lib/*.jar) to an IntelliJ IDEA project
to use the package.
This works, but distributing a jar does not feel nice security-wise. On the other hand, this discussion
IntelliJ IDEA - adding .java file to project dependencies
suggests that to use the source code, you need to copy it into your src folder.
Is there a way to distribute source code (java files) only so that if multiple projects use the same package
the package only needs to be downloaded once
the package can be kept up to date with git pull?
I would really recommend not include jar or any binaries in a Git repo and the best approach to keep these dependencies in a local Nexus repository and use maven or Gradle as your dependency management tool.
I found a working solution:
Supply an Ant build file with the package. The build file compiles classes and packages them into a jar file. The default target is building the jar, which depends on compiling the classes.
Provide users with instructions on how to set the given Ant build file as a build file in Intellij IDEA and build the default target.
Then instruct them to follow the steps in the first link above to add the jar as a dependency.
I have an existing codebase where a number of third-party dependencies were added as Git submodules, and their directories are directly referenced inside CMakeLists.txt, as in include_directories(../external/foo).
Some of the dependencies are large projects, like FFmpeg, and I'd rather just use the version I installed to my system with a package manager. But the maintainer of the codebase wants to be able to continue using the in-tree dependencies.
I thought a compromise would be to configure CMake to permit both, either using an installed package, or using the in-tree submodules. I think find_package can be used to find the installed package, but is there a good way to implement this behavior that isn't too hacky?
You can add an option to your cmake file that allows the user to switch between the internal FFMpeg or the system one. option(INTREE_FFMPEG "Use the intree ffmpeg" ON). The option can be set either via the cmake-gui or as a command line switch.
I run: conan install Boost/1.64.0#conan/stable, and it fails.
Output:
C:\temp>conan install Boost/1.64.0#conan/stable
Boost/1.64.0#conan/stable: Not found in local cache, looking in remotes...
Boost/1.64.0#conan/stable: Trying with 'bintray'...
Boost/1.64.0#conan/stable: Trying with 'conan.io'...
ERROR: Unable to find 'Boost/1.64.0#conan/stable' in remotes
Trying other package, works:
C:\temp>conan install fmt/4.0.0#bincrafters/stable
fmt/4.0.0#bincrafters/stable: Not found in local cache, looking in remotes...
fmt/4.0.0#bincrafters/stable: Trying with 'bintray'...
fmt/4.0.0#bincrafters/stable: Trying with 'conan.io'...
Downloading conanmanifest.txt
[==================================================] 121B/121B
Downloading conanfile.py
[==================================================] 1.8KB/1.8KB
fmt/4.0.0#bincrafters/stable: Installing package
Requirements
fmt/4.0.0#bincrafters/stable from conan.io
Packages
fmt/4.0.0#bincrafters/stable:63da998e3642b50bee33f4449826b2d623661505
fmt/4.0.0#bincrafters/stable: Retrieving package 63da998e3642b50bee33f4449826b2d623661505
fmt/4.0.0#bincrafters/stable: Looking for package 63da998e3642b50bee33f4449826b2d623661505 in remote 'conan.io'
Downloading conanmanifest.txt
[==================================================] 938B/938B
Downloading conaninfo.txt
[==================================================] 491B/491B
Downloading conan_package.tgz
[==================================================] 159.8KB/159.8KB
fmt/4.0.0#bincrafters/stable: Package installed 63da998e3642b50bee33f4449826b2d623661505
Any idea why the package isn't found?
How to debug it?
Conan is a decentralized package manager (kind of git-like style), so it can have many remotes. By default it comes configured with 2 remotes:
conan-transit: Is a read-only copy of the old conan.io repository, which contains many different Boost packages, from different authors. Quality is variable, so some packages might work only for certain OS, or might fail for some configurations.
conan-center: It is a moderated/reviewed repository, package creators can submit inclusion requests to share their packages with the community.
So far conan-transit contains several Boost/1.64 packages, so can check it with:
$ conan search Boost* -r=conan-transit
$ conan search Boost* -r=conan-center
As you can see the package you are trying to install doesn't exist in these repositories.
As I said above, conan is decentralized, so you can use different remotes. For example, the "bincrafters" community has a bintray repo that can be added with:
$ conan remote add bincrafters https://api.bintray.com/conan/bincrafters/public-conan
$ conan search Boost* -r=bincrafters
You will see they have a large number of Boost/1.64 packages, because they have created a modularized version of boost, in which every library lives in a different package, so you only get installed what you need.
UPDATE: Packages in the central repository are being renamed by the community to lowercase. Try with boost lowercase in the above if necessary.