Context
I'd like to run a Qt application on an IM6 based system with a Yocto built image. I already have the meta-qt5 layer in my project. I started with a simple Qt5 application that only neededs the following modules:
QT += core gui widgets
All I have to do is make sure my bitbake recipe has DEPENDS += qtbase and is based on the qmake class with: inherit qmake5. And it builds and runs on the target! No problem
Problem
Now I'd like to add another Qt5 application, this time with the following modules and one plugin:
QT += core gui widgets quick qml svg xml network charts
QTPLUGIN += qsvg
Unfortunately, I'm not able to simple add these to my DEPENDS variable and get it to work. But googling around for how to add support reveals what seems to be a sprawling assortment of solutions. I'll enumerate what I've found here:
I need to add inherit populate_sdk_qt5 to instruct bitbake to build the recipe against the SDK that contains the libraries for the modules (see here)
I need to add IMAGE_FEATURES += dev-pkgs to the recipe (see here)
I need to modify local.conf for the system, and add lines like: PACKAGECONFIG_append_pn_qttools = "..." and also PACKAGECONFIG_append_pn-qtbase = "..."
I need to modify layer.conf in my layer and add things like IMAGE_INSTALL_append = "qtbase qtquick ..." (slide 53 here)
I need to manually patch the Qt5 toolchain for charts? (see here)
I need to compile my image using bitbake <target> -c populate_sdk? (see here again)
At this point, I'm really unsure what exactly is going on. It seems we're modifying the recipe, the layer configuration file, the distribution configuration file, and even meta-Qt layer files. I know that I fundamentally need to do a few things:
Compile the application against the Qt5 SDK
Compile the needed plugins + modules for the target architecture
Make sure the appropriate binaries (images) are copied to the target.
But it has become a bit unclear about what does what. I know that IMAGE_INSTALL_append adds images to the target, but I am lost with what is the proper way to add the modules. I don't want to go about randomly adding lines, so I'm hoping someone can clear up a bit what exactly I need to be looking at in order to add support for a Qt5 module for an application.
There are different problems stated, your preferred way seems to the directly building a recipe, not using the toolchain. So, you need the image to have the tools you need.
First of all qtsvg is not on Qt Base, it is a module so you need it installed.
Add Qt SVG support
You need Qt SVG on target in order to run your App. Either to your image or to local.conf you need
IMAGE_INSTALL_append = " qtsvg"
As a fact, your app's recipe needs Qt QSVG so you need to DEPEND on it on your app's recipe like this:
DEPENDS = "qtsvg"
here qtsvg is the name of the other recipe, namely qtsvg_git.bb, not to be confused with the identically named qtsvg plugin. And it will get pulled automatically on build time on your development machine, otherwise it won't even build.
Remember yocto creates a simulated image tree on the TMP folder in order to build (yes it does it for each recipe), so you must describe what your recipe needs or it won't find it and your build will fail.
You can also check the recipe for a Qt5 example as it also has DEPENDS and RDEPENDS. And you can get more info on those in here.
Related
Context
I'm currently working on yocto on a build for raspberrypi 4 (not really relevant). I would like to create two recipes for two different images:
An image with the bare minimum to run fast. Mostly for production phase
An image with a lot of extra tools and debug options
Basically the core idea is pretty easy to sort out, I just create two image files, the first one requires the basic recipes for my image, the second one requires the first file and requires a few additional modules (such as apt, valgrind, gdb, etc...).
Question
Now here is the issue: Most of my recipes are built with cmake. I would like to add extra flags (especially -g) when building for the debug image.
What is the best way or most common way to do that? Is there a good practice?
To me this is a little tricky due to the fact that bitbake does not supports the if statement.
My solution
For now here is the solution I found: In the image recipe for the dev file add a variable to define the debug mode:
DEBUG_MODE = "1"
Then in every file that compiles a module with cmake conditionally add the flag (-g for example):
TARGET_CXXFLAGS += "${#'-g' if d.getVar('DEBUG_MODE') == '1' else ''}"
However I'm not sure how this apply to dependencies, and I'm not sure if it is the best way to do it. It seems like a lot of duplicate code and tedious to maintain. Is there anything easier to manage?
Currently you set
DEBUG_MODE = "1"
but then check for DEV_MODE in your if condition, so this approach should not work at all.
Suggestion: Set -g globally
One step could be to define TARGET_CPPFLAGS in a *.conf for your debug-image to include -g.
TARGET_CPPFLAGS += "-g"
That way there should be no need to add it on recipe base, but can be done globally. Also when it is added in a conf only used for your debug image, there is no need for any if/else.
I would also use TARGET_CPPFLAGS, as those are then used for c and c++. But bear in min that some recipes might unset the compiler flags again.
Please allow me two questions to the use in Conan.io in our environment:
We are developing automotive embedded software. Usually, this includes integration of COTS libraries, most of all for communication and OS like AUTOSAR. These are provided in source code. Typical uC are Renesas RH850, RL78, or similar devices from NXP, Cypress, Infinion, and so on. We use gnumake (MinGW), Jenkins for CI, and have our own EclipseCDT distribution as standardized IDE.
My first question:
Those 3rd party components are usually full of conditional compilation to do a proper compile-time configuration. With this approach, the code and so the resulting binaries are optimized, both in size and in run-time behavior.
Besides those components, we of course have internal reusable components for different purposes. The Compile-time configuration here is not as heavy as in the above example, but still present.
In one sentence: we have a lot of compile-time configuration - what could be a good approach to set up a JFrog / Conan based environment? Stay with the sources in every project?
XRef with Conan:
Is there a way to maintain cross-reference information coming from Conan? I am looking for something like "Project xxx is using Library lll Version vvv". In that way, we would be able to automatically identify other "users" of a library in case a problem is detected.
Thanks a lot,
Stefan
Conan recipes are based on python and thus are very flexible, being able to implement any conditional logic that you might need.
As an example, the libxslt recipe in ConanCenter contains something like:
def build(self):
self._patch_sources()
if self._is_msvc:
self._build_windows()
else:
self._build_with_configure()
And following this example, the autotools build contains code like:
def _build_with_configure(self):
env_build = AutoToolsBuildEnvironment(self, win_bash=tools.os_info.is_windows)
full_install_subfolder = tools.unix_path(self.package_folder)
# fix rpath
if self.settings.os == "Macos":
tools.replace_in_file(os.path.join(self._full_source_subfolder, "configure"), r"-install_name \$rpath/", "-install_name ")
configure_args = ['--with-python=no', '--prefix=%s' % full_install_subfolder]
if self.options.shared:
configure_args.extend(['--enable-shared', '--disable-static'])
else:
configure_args.extend(['--enable-static', '--disable-shared'])
So Conan is able to implement any compile time configuration. That doesn't mean that you need to build always from sources. The parametrization of the build is basically:
Settings: for "project wide" configuration, like the OS or the architecture. Settings values typically have the same value for all dependencies
Options: for package specific configuration, like a library being static or shared. Every package can have its own value, different to other packages.
You can implement the variability model for a package with settings and options, build the most used binaries. When a given variant is requested, Conan will error saying there is not precompiled binary for that configuration. Users can specify --build=missing to build those from sources.
I'm in the process of rewriting a legacy CMake setup to use modern features like automatic dependency propagation. (i.e. using things like target_include_directories(<target> PUBLIC <dir>) instead of include_directories(<dir>).) Currently, we manually handle all project dependency information by setting a bunch of global directory properties.
In my testing, I've found a few examples where a target in the new build will link to a library that the old build would not. I'm not linking to it explicitly, so I know this is coming from the target's dependencies, but in order to find which one(s) I have to recursively look through all of the project's CMakeLists.txts, following up the dependency hierarchy until I find one that pulls in the library in question. We have dozens of libraries so this is not a trivial process.
Does CMake provide any way to see, for each target, which of its dependencies were added explicitly, and which ones were propagated through transitive dependencies?
It looks like the --graphviz output does show this distinction, so clearly CMake knows the context internally. However, I'd like to write a tree-like script to show dependency information on the command line, and parsing Graphviz files sounds like both a nightmare and a hack.
As far as I can tell, cmake-file-api does not include this information. I thought the codemodel/target/dependencies field might work, but it lists both local and transitive dependencies mixed together. And the backtrace field of each dependency only ties back to the add_executable/add_library call for the current target.
You can parse dot file generated by graphviz and extract details which you want. Below is sample python script to do that.
import pydot
import sys
graph = pydot.graph_from_dot_file(sys.argv[1])
result = {}
for g in graph:
# print(g)
for node in g.get_node_list():
if node.get("label") != None:
result[node.get("label")] = []
for edge in g.get_edges():
result[g.get_node(edge.get_source())[0].get("label")].append(g.get_node(edge.get_destination())[0].get("label"))
for r in result:
print(r+":"+",".join(result[r]))
You can also add this script to run from cmake as custom target, so you can call it from you build system. You can find sample cmake project here
I have this project that I have done for experimentation with Qt and shared libraries. This is basically a couple of Qt Widgets from the tutorials for Qt and what I think is the right CMakeLists configuration so a MylibConfig.cmake is automatically generated from a MylibConfig.cmake.in to share the library. The problem is that I don't want the end user to add the dependencies of my library to its own CMakeLists.txt. This is, in my case, the library depends on Qt4, but I want that the end user to not have to do find_package(Qt 4 REQUIRED). Imagine that I want to provide an enclosed functionality to someone that does not need or want to know about what my library is built on. Is there a way in the automatic generation of the MylibConfig.cmake that it automatically finds all necessary packages or is the only option to add the fin package manually in the MylibConfig.cmake.in?
Thank you very much.
In fact, both mentioned projects do find of dependencies from their *Config.cmake files. And nowadays that is the only option -- CMake can't help you to do it "automatically".
So, some way or another, your config module should do the same.
The easy way is to add find_dependency() calls (cuz you know exactly what other packages, your project is based on).
A little bit harder is to do it "automatically" (writing your own helper function) -- for example by inspecting properties of your target(s), "searching" where all that libraries come from and finally generating find_dependency() calls anyway.
Is there any way to make CMake "forget" about a file in the dependency tree? My original problem (to avoid the XY situation) is the following: I want to timestamp the build of a set of tools which have complicated dependencies among them and to other tools. Right now, I want to use a pure timestamp, but later I might want add some info from the repository (SVN). Whatever system I end up implementing needs to have the following characteristics (my "X"):
No unnecessary rebuilding: the executables should not be rebuilt on every make if the only change would be the timestamp.
Update on any change: if any tool is going to be rebuilt or relinked, either by changes to its code or to one of its dependencies, the timestamp needs to be updated.
My current solution goes along the lines of creating a custom command+target that invokes CMake at make time (so the command calls CMake itself with -P script.cmake) to generate a timestamp.h file. The main files of my tools would include that file, and the projects would depend on the target so that it gets rebuilt first.
However, this has its drawbacks: if I do update the timestamp file on every call to make, then CMake's dependency scanner would know about that file even if I do not list it as an explicit dependency of my tools. Thus, every make would trigger at least a recompilation of the respective "main" files and the corresponding relink. With tens of tools, this means slowing down the build when I may be working on just two or three of them at once.
So, I was thinking that my solution would be to somehow make CMake forget about that file when building its dependency tree for the "main" file of each tool. I would keep the dependency on the custom target that does depend on the file, so that it would be regenerated first on each call to make. However, the build tool would not consider that file as relevant to determine whether it is necessary to actually rebuild each individual tool. Thus, tools only with other changes would be rebuilt (satisfying my first criterion), and any change that causes a rebuild of a tool would obviously use the version just generated (fulfilling the second criterion).
To my chagrin, I have not found a way to make the dependency scanner forget about this file, so my solution cannot be put to use. How would I go about doing such a thing? Is it even possible, or is it completely the wrong way to go about this? I am using CMake 3.4, and my code is currently C++, but I would like a solution that did not rely on C/C++ specifics, since I have a different project (written in Fortran) in which I would also like to have build timestamping.
I've had almost the same problem than you are. Simply solved by pushing the timestamp header file into standalone target containing only this header generator command. After that you have several choices:
1.. Exclude that project from the build by the IDE you are using. For example, for the Visual Studio you can do it by several ways:
1.1. Project->Project Dependencies...->uncheck project with that header (not always works: Error while removing project dependency in VS2010)
1.2. Build->Configuration Manager...->uncheck project with that header
2.. Create an environment variable and use the condition with that variable around the add_dependencies command in the CMakeLists.txt file.
3.. Generate 2 standalone solutions through the cmake generator with included and with excluded add_dependencies in the CMakeLists.txt file.
I've used particulary [1.2]. When i need build and debug, then i uncheck the dependecy. By default, dependecy always checked, so there is no problem to miss timestamp build for a build server.
Note:
The timestamp header will be included in all projects you want to include that header (for example, through the add_library and add_executable) and you still can observe it in the IDE under a project item menu even if a project depends on the timestamp project indirectly. This is useful if you don't want to search for the timestamp project with the header to open it from there and want to open it from any project which has included that header.
So, in case of removing the timestamp header from the add_library or add_executable you won't have that opportunity.