How do I build a debian package from source using bazel?
I am trying to build debian packagefor tensorflow. I need that to be included in our PPA server. Thanks!
building Debian packages consists of compiling the software (mostly; there are also packages that don't need compilation, e.g. for scripting languages), and then packaging the artifacts.
Therefore the packaging process has a separate "build" step, which is used to trigger your software's build process.
This step doesn't care whether you use make, CMake, SCons, bazel or whatever, as long as you tell it what it should do.
a simplistic debian/rules file for your needs could look like:
#!/usr/bin/make -f
%:
dh $#
override_dh_auto_build:
bazel build //main:hello-world
But of course there is quite a lot to Debian packaging in general, so you make sure you read (and understand) the Debian Packaging Documentation first...
Related
I am considering to create a Debian package from an existing library (paho-mqtt-c). The project uses CMake as its build system. After some research I think I need to create two or three different packages:
libpaho-mqtt3 (with library .so files and related stuff)
libpaho-mqtt3-dev (with header files)
also maybe I need a third package with sample files or documentation (called paho-mqtt3?)
I have done some research and it seems there exist at least three different ways how I can create a Debian package when I use CMake as my build system:
Use debmake procedure described in Debian documentation (Chapter 8).
Use cmake-debhelper.
Use dh-cmake
I have looked into all three methods and it seems each has some advantages and disadvantages.
Debmake
As far as I have understood using debmake assumes I have an upstream tarball with the sources and the build system and then I invoke debmake on the extracted tarball. Afterwards I get a lot of templates which I need to manually adjust to fill in the missing gaps. I started doing this but it seems quite complex.
cmake-debhelper
I tried to use it but received lots of errors. The github page has an open issue with no solution so I stopped looking at this. This is also what the paho-mqtt-c build system is currently using, but it does not work due to the issue linked.
dh-cmake
I briefly looked into this and it seems to be the most modern solution and it should be possible to combine this with CPack. However, it seems dh-cmake is only available for Ubuntu 18.04 and 16.04, but I am using Ubuntu 19.10 so I was not able to install dh-cmake on my system.
Have I missed anything in my research? What are the recommended steps to create a Debian package from a software managed with CMake and which documentation is useful to read?
In short, on Ubuntu you need to create at least these files:
debian/
changelog
control
copyright
rules
And then run debuild and it will run cmake install to temporary folder and pack an installable deb package from it.
To quickly create those debian files run dh_make --createorig and press s for source package.
Then you'll need to carefully edit debian files as described in Chapter 4. Required files under the debian directory
of Debian New Maintainers' Guide.
If you need to set cmake properties or make any other configuration then you'll need to adjust override_dh_auto_configure in rules:
#!/usr/bin/make -f
# See debhelper(7) (uncomment to enable)
export DH_VERBOSE = 1
%:
dh $#
override_dh_auto_configure:
dh_auto_configure -- \
-DCMAKE_LIBRARY_PATH=$(DEB_HOST_MULTIARCH) \
-DIWINFO_SUPPORT=OFF
Here the -DCMAKE_LIBRARY_PATH=$(DEB_HOST_MULTIARCH) and -DIWINFO_SUPPORT=OFF will be directly passed to cmake.
You can then upload your package to Ubuntu PPA:
debuild -S -I
dput dput ppa:your-launchpad-user/your-ppa ../*_source.changes
After that PPA build bot will compile and publish your package to PPA and you'll see them on https://launchpad.net/~your-launchpad-user/+archive/ubuntu/your-ppa/+packages
Unfortunately there is a lot of other steps, I just described briefly.
The dh-cmake is needed for more sophisticated things. CPack won't work for you if you want to publish to PPA because its buildbot will anyway run debhelper (short version of debuild) so it needs for the debian folder
or you could use cpack with cmake to generate a deb fairly easy to do but cmake and cpack are poorly documented still they work well
I suggest adding the following to the bottom of CMakeLists.txt
# generate postinst file in ${CMAKE_BINARY_DIR} from template #
CONFIGURE_FILE("${CMAKE_SOURCE_DIR}/contrib/postinst.in" "postinst" #ONLY IMMEDIATE)
# generate a DEB when cpack is run
SET(CPACK_GENERATOR "DEB")
SET(CPACK_PACKAGE_NAME ${CMAKE_PROJECT_NAME})
SET(CPACK_SET_DESTDIR TRUE)
SET(CPACK_DEBIAN_PACKAGE_MAINTAINER "grizzlysmit#smit.id.au")
SET(CPACK_PACKAGE_VERSION_MAJOR "0")
SET(CPACK_PACKAGE_VERSION_MINOR "0")
SET(CPACK_PACKAGE_VERSION_PATCH "1")
include(GNUInstallDirs)
SET(CPACK_PACKAGE_DESCRIPTION_FILE "${CMAKE_SOURCE_DIR}/docs/CPack.Description.txt")
SET(CPACK_RESOURCE_FILE_README "${CMAKE_SOURCE_DIR}/docs/README.md")
SET(CPACK_RESOURCE_FILE_LICENSE "${CMAKE_SOURCE_DIR}/docs/LICENCE")
SET(CPACK_DEBIAN_PACKAGE_DEPENDS "libreadline8, libreadline-dev")
SET(CPACK_PACKAGE_VENDOR "Grizzly")
# make postinst run after install #
SET(CPACK_DEBIAN_PACKAGE_CONTROL_EXTRA "${CMAKE_BINARY_DIR}/postinst;")
include(CPack)
the postisnt is to run a script after the install see CMAKE/CPACK:I want to the deb executes a bash script after installed, but it doesn't work for more on that.
A quote from the official documentation:
"Specify rules to run at install time."
What exactly is install time?
The problem for me: I am on Linux, software is installed from packages that are just dependencies and data. There is no CMake that can do anything here. So installation time of software is out of scope from CMake. So what exactly do they mean?
Building a CMake project can roughly be divided into three phases:
Configure time. This includes everything that happens while running cmake itself. This phase is concerned with inspecting certain properties of the host system and generating the specific build files for that platform under the selected configuration.
Build time. This includes everything that happens while actually building your project from the files generated by CMake (like, when running cmake --build or make). This is where all of the actual compilation and linking happens, so at the end of the build phase, you have a usable binary.
Install time. This includes everything that happens when running the INSTALL target generated by CMake (like, when running cmake --build --target install or make install). This takes care of copying the binaries that were generated into the build tree to a different directory. Note that the build tree contains a lot of stuff that is no longer needed once the build is completed if you are only interested in running the binary. Examples include all intermediate build artifacts, like the build files generated during the configure phase or the intermediate object files created during the build phase. Furthermore, the install phase might include additional steps to ensure that the binaries produced during the build are portable. For instance, on Linux systems you might want to remove the build directory from the shared library search path in the binary and replace it with a portable equivalent. So the install phase might do more than just copy all the important files to a new directory. It could also include additional steps that change the binaries to make them more portable.
Note that the last phase is optional. If you do not want to support calling make install but prefer another deployment mechanism, you simply don't use the install command in your CMake script and no INSTALL target will be generated.
I'd like to expand the answer, which ComicSansMS gave you, a little bit.
As he mentioned - CMake generates an extra target called install for the make tool (when you use a Makefile-based generator).
It may look weird for you as a package system is used for Linux. However the install target is still useful or even necessary:
When you develop your application you may need to install (move binaries and possibly some include files) to a certain location so some of your projects may see each other. For example, you may develop a library and a set of non-related applications which use it. Then this library must be installed somewhere to be visible. It doesn't mean you need to put it to the /usr directory; you may use your /home.
The process of Linux package preparation requires an install step. For example, the RPM packaging system does three main steps when the rpm package file is being built: the project is configured, then is compiled and linked and finally is being installed to a certain location. All files from this location are being packed to the rpm file.
I started to learn Rust programming language and I use Linux. I'd like to build a cross-platform application using this language.
The question might not be related to Rust language in particular, but nonetheless, how do I do that? I'm interested in building a "Hello World" cross-platform application as well as for more complicated ones. I just need to get the idea.
So what do I do?
UPDATE:
What I want to do is the ability to run a program on 3 different platforms without changing the sources. Do I have to build a new binary file for each platform from the sources? Just like I could do in C
To run on multiple platforms you need to build an executable for each as #huon-dbauapp commented.
This is fairly straightforward with Rust. You use "--target=" with rustc to tell it what you want to build. The same flag works with Cargo.
For example, this builds for an ARM target:
cargo build --target=arm-unknown-linux-gnueabihf
See the Rust Flexible Target Specification for more about targets.
However, Rust doesn't ship with the std Crate compiled for ARM (as of June 2015). If this is the case for your target, you'll first need to compile the std Crates for the target yourself, which involves compiling the Rust compiler from source, and specifying the target for that build!
For information, most of this is copied from: https://github.com/japaric/ruststrap/blob/master/1-how-to-cross-compile.md
The following instructions are for gcc, so if you don't have this you'll need to install it. You'll also need the corresponding cross compiler tools, so for gcc:
sudo apt-get install gcc-arm-linux-gnueabihf
Compile Rust std Crate For ARM
The following example assumes you've already installed the current Rust Nightly, so we'll just get the sources and compile for ARM. If you are using a different version of the compiler, you'll need to get that to ensure your ARM libraries match the version of the compiler you're using to build your projects.
mkdir ~/toolchains
cd ~/toolchains
git clone https://github.com/rust-lang/rust.git
cd rust
git update
Build rustc for ARM
cd ~/toolchains/rust
./configure --target=arm-unknown-linux-gnueabihf,x86_64-unknown-linux-gnu
make -j4
sudo make install
Note "-j4" needs at least 8GB RAM, so if you hit a problem above try "make" instead.
Install ARM rustc libraries In native rustc build
sudo ln -s $HOME/src/rust/arm-unknown-linux-gnueabihf /usr/lib/rustlib/arm-unknown-linux-gnueabihf
Create hello.rs containing:
pub fn main() {
println!("Hello, world!");
}
Compile hello.rs, and tell rustc the name of the cross-compiler (which must be in your PATH):
rustc -C linker=arm-linux-gnueabihf-gcc-4.9 --target=arm-unknown-linux-gnueabihf hello.rs
Check that the produced binary is really an ARM binary:
$ file hello
hello: ELF 32-bit LSB shared object, ARM, EABI5 version 1 (SYSV), (..)
SUCCESS!!!:
Check: the binary should work on an ARM device
$ scp hello me#arm:~
$ ssh me#arm ./hello
Hello, world!
I've used this to build and link a Rust project with a separate C library as well. Instructions similar to the above on how to do this, dynamically or statically are in a separate post, but I've used my link quota up already!
The best way to figure this out is to download the source code for Servo and explore it on your own. Servo is absolutely a cross-platform codebase, so it will have to address all of these questions, whether they be answered in build/configuration files, or the Rust source itself.
It looks like the rust compiler might not be ready to build standalone binaries for windows yet (see the windows section here), so this probably can't be done yet.
For posix systems it should mostly Just Work unless you're trying to do GUI stuff.
Yes, you won't need to change the source, unless you are using specific libraries that are not cross-platform.
But as #dbaupp said native executables are different on each platform, *nix uses ELF, Windows PE, and OSX Mach-O. So you will need to compile it for each platform.
I don't know the state of cross-compiling in rust, but if they already implemented it, then you should be able to build all the binaries in the same platform, if not, you will have to build each binary on it's platform.
I'm thinking to write a simple configure script (similar to autoconf one) which execs cmake. But before doing that I want to check if anyone knows of such an effort already. I wasn't able to find anything on google.
It should be able to support the basic autoconf configure flags (prefix, exec-prefix, bindir mostly).
Reason to do it is of course that there's a certain user expectancy to be able to do ./configure && make
Also not really an answer but too long for a comment:
After reading up about cmake / cpack, I can at least tell you this. Cmake expects to be present on the platform. Therefore CPack cannot generate the same type of ./configure scripts as autotools. The Autotools expect some shell to be present, which is essentially the same as cmake to be present. However since cmake also targets the Win environment, it cannot rely on a shell. That being said, CPack can provide source packages, which need to be installed with cmake in the usual manner.
Also this does not solve your problem, I do not recommend to write a tool for cmake. Cmake is able to use all these type of prefixes you are interested in. If the user wants to compile your program from scratch, he has to know at least the basics (e.g. setting variables) of your build system. This is also true for autotools. If you want to spare him the pain, you can provide binary .sh, .deb or .rpm packages, which can be easily built with cmake / cpack.
In short I'm trying to cross compile CMake with CMake, and I don't think I'm linking libraries correctly. What I want to do may not be possible, but I'd at least like to know why it isn't possible if that's the case.
System: The host is a Linux box with a Cavium ARM9 CPU. It's currently running version 2.6.24.4 of the Linux kernel and Debian 5.0 (Lenny). My workstation is a Core i5 running Ubuntu 12.04 LTS (Precise Pangolin).
My overall goal is to get ROS running on the Linux box. I have to compile from source rather than use apt since Debian 6.0 (Squeeze) binaries require thumb support that the Cavium does not give, and not many of the needed packages are available for Debian 5.0 (Lenny). I'd made progress installing the various libraries needed, but when I got to step 1.3.1 and tried to run CMake, I got the error
CMake 2.8 or higher is required. You are running version 2.6.0
Next I tried to download and build CMake 2.8.8 on the Linux box itself, but it was too much for the system. When that failed, I downloaded the toolchain suggested on the manufacturer's website and used the cross-compiling guide at [www.cmake.org/Wiki/CMake_Cross_Compiling] to build the CMake executables. Here is my toolchain file:
# This one is important
SET(CMAKE_SYSTEM_NAME Linux)
# Specify the cross compiler
SET(CMAKE_C_COMPILER /pathto/crosstool-linux-gcc-4.5.2-gclibc-2.9-oabi/arm-unknown-linux-gnu/bin/arm-unknown-linux-gnu-gcc)
SET(CMAKE_CXX_COMPILER /pathto/crosstool-linux-gcc-4.5.2-gclibc-2.9-oabi/arm-unknown-linux-gnu/bin/arm-unknown-linux-gnu-g++)
# Where is the target environment
SET(CMAKE_FIND_ROOT_PATH /pathto/crosstool-linux-gcc-4.5.2-gclibc-2.9-oabi/arm-unknown-linux-gnu /pathto/crosstool-linux-gcc-4.5.2-gclibc-2.9-oabi/arm-unknown-linux-gnu/arm-unknown-linux-gnu)
# Search for programs in the build host directories
SET(CMAKE_FIND_ROOT_PATH_MODE_PROGRAM NEVER)
# For libraries and headers in the target directories
SET(CMAKE_FIND_ROOT_PATH_MODE_LIBRARY ONLY)
SET(CMAKE_FIND_ROOT_PATH_MODE_INCLUDE ONLY)
However, use of the binary on the Linux box gives the error
cmake: /usr/lib/libstdc++.so.6: version `GLIBCXX_3.4.14' not found (required by cmake)
Sure enough, the library is not there:
prompt# strings /usr/lib/libstdc++.so.6 | grep GLIBC
GLIBCXX_3.4
GLIBCXX_3.4.1
GLIBCXX_3.4.2
GLIBCXX_3.4.3
GLIBCXX_3.4.4
GLIBCXX_3.4.5
GLIBCXX_3.4.6
GLIBCXX_3.4.7
GLIBCXX_3.4.8
GLIBCXX_3.4.9
GLIBCXX_3.4.10
GLIBC_2.3
GLIBC_2.0
GLIBC_2.3.2
GLIBC_2.1
GLIBC_2.1.3
GLIBC_2.2
GLIBCXX_FORCE_NEW
GLIBCXX_DEBUG_MESSAGE_LENGTH
I've never cross-compiled before, but I can see one of two scenarios happening: either the binary got created with a link to a higher version of glibcxx on the host machine or the manufacturer's toolchain is more modern than their image. I don't know how to check which is happening or if something else is happening that I don't know about.
My last effort involved trying to statically cross-compile CMake to hopefully get rid of the linking error with
cmake -DCMAKE_TOOLCHAIN_FILE=../toolchain-technologic.cmake -DBUILD_SHARED_LIBS=OFF -DCMAKE_BUILD_TYPE=Release -DCMAKE_EXE_LINKER_FLAGS_RELEASE="-static" ..
I got build errors, and that binary didn't work either. I got:
FATAL: kernel too old
Segmentation fault
I'd try installing glibcxx 3.4.14 on the Linux box, but it doesn't look like it's available for this processor.
I've tried searching for CMake dependencies or system requirements and can't find anything. I've also searched on how to build CMake, but most searches turn up how to build other things with CMake rather than building CMake itself.
I do cross-compile a lot for ARM9 devices using CMake, and indeed this looks like you're not linking to the same libs you have on your target device. You shouldn't need to build CMake yourself to get this done, since it does have good support for cross-compiling since version 2.6. Just make sure you set the CMAKE_FIND_ROOT_PATH variable to a path where you have an exact copy of the root filesystem you have on your target device (with libraries and binaries pre-compiled for the target processor). That should solve your problems.
As a sidenote, I like to use crosstool-ng for building my cross-compilers. It is a really nice tool which helps you to build them from scratch, so I try to match the compiler version and glibc to the ones originally used to build the root filesystem (I usually start with a ready made root filesystem from ARMedslack, since I use Slackware for my development box and ARMedslack for my ARM targets).