how to customize nix package builder script - cmake

The root problem is that nix uses autoconf to build libxml2-2.9.14 instead of cmake, and a consequence of this is that the cmake-configuration is missing (details like version number, platform specific dependencies like ws2_32 etc which are needed by my project cmake scripts). libxml2-2.9.14 already comes with cmake configuration and works nicely, except that nix does not use it (I guess they have their own reasons).
Therefore I would like to reuse the libxml2-2.9.14 nix package and override the builder script with my own (which is a trivial cmake dance).
Here is my attempt:
defaultPackage = forAllSystems (system:
let
pkgs = nixpkgsFor.${system};
cmakeLibxml = pkgs.libxml2.overrideAttrs( o: rec {
PROJECT_ROOT = builtins.getEnv "PWD";
builder = "${PROJECT_ROOT}/nix-libxml2-builder.sh";
});
in
Where nix-libxml2-builder.sh is my script calling cmake with all the options I need. It fails like this:
last 1 log lines:
> bash: /nix-libxml2-builder.sh: No such file or directory
For full logs, run 'nix log /nix/store/andvld0jy9zxrscxyk96psal631awp01-libxml2-2.9.14.drv'.
As you can see the issue is that PROJECT_ROOT does not get set (ignored) and I do not know how to feed my builder script.
What am I doing wrong?

Guessing from the use of defaultPackage in your snippet, you use flakes. Flakes are evaluated in pure evaluation mode, which means there is no way to influence the build from outside. Hence, getEnv always returns an empty string (unfortunately, this is not properly documented).
There is no need to refer to the builder script via $PWD. The whole flake is copied to the nix store so you can use your files directly. For example:
builder = ./nix-libxml2-builder.sh;
That said, the build will probably still fail, because cmake will not be available in the build environment. You would have to override nativeBuildInputs attribute to add cmake there.

Related

How to glob an absolute path for files in Bazel

Context: I'm trying to come up with a fix for https://github.com/tensorflow/tensorflow/issues/37861 where header files of an external dependency are manually listed but that list is version specific and hence impossible to keep up to date.
What is happening:
tf_http_archive(name = "com_google_protobuf", system_build_file = clean_dep("//third_party/systemlibs:protobuf.BUILD") ...) is invoked
tf_http_archive is a repository_rule with effectively nothing but ctx.template("BUILD.bazel", ctx.attr.system_build_file, {...}, False)
In the protobuf.BUILD there is a list HEADERS = ["google/protobuf/any.pb.h", ...] which is passed to the hdrs argument of cc_library calls
a genrule apperantly symlinks those headers from $(INCLUDEDIR) into $(#D) (I'm not really familiar with Bazel but IIUC the latter is some internal build directory used later)
As I'm unfamiliar with Bazel in general I'll just assume the list of headers is required and there exists a $(INCLUDEDIR)/google/protobuf folder and is somewhere (else) on the system, e.g. /usr/local/include.
Is there any way to get all *.h and *.inc files in the format (i.e. relative to $(INCLUDEDIR)) via a glob or similar? The Bazel glob function doesn't work for absolute paths, so that can't be used.
I found https://github.com/bazelbuild/bazel/issues/8846 suggesting to use new_local_repository with a build_file and a path set to (in this case) $(INCLUDEDIR) but I don't see how that could be applied to the tf_http_archive (which has some conditions to either download the dependency or just use the system_build_file). This seems to also allow to avoid the symlinking (which I'm highly suspicious of anyway because that folder is added via -iquote but include style is #include <...>, see my comments in https://github.com/tensorflow/tensorflow/issues/37861)
Bonus points for people contributing to the issue or ideas why action_env environment variables seem to be ignored in a native.cc_library call.

Meson equivalent of automake's CONFIG_STATUS_DEPENDENCIES?

I have a project whose build options are complicated enough that I have to run several external scripts during the configuration process. If these scripts, or the files that they read, are changed, then configuration needs to be re-run.
Currently the project uses Autotools, and I can express this requirement using the CONFIG_STATUS_DEPENDENCIES variable. I'm experimenting with porting the build process to Meson and I can't find an equivalent. Is there currently an equivalent, or do I need to file a feature request?
For concreteness, a snippet of the meson.build in progress:
pymod = import('python')
python = pymod.find_installation('python3')
svf_script = files('scripts/compute-symver-floor')
svf = run_command(python, svf_script, files('lib'),
host_machine.system())
if svf.returncode() == 0
svf_results = svf.stdout().split('\n')
SYMVER_FLOOR = svf_results[0].strip()
SYMVER_FILE = svf_results[2].strip()
else
error(svf.stderr())
endif
# next line is a fake API expressing the thing I can't figure out how to do
meson.rerun_configuration_if_files_change(svf_script, SYMVER_FILE)
This is what custom_target() is for.
Minimal example
svf_script = files('svf_script.sh')
svf_depends = files('config_data_1', 'config_data_2') # files that svf_script.sh reads
svf = custom_target('svf_config', command: svf_script, depend_files: svf_depends, build_by_default: true, output: 'fake')
This creates a custom target named svf_config. When out of date, it runs the svf_script command. It depends on the files in the svf_depends file object, as well as
all the files listed in the command keyword argument (i.e. the script itself).
You can also specify other targets as dependencies using the depends keyword argument.
output is set to 'fake' to stop meson from complaining about a missing output keyword argument. Make sure that there is a file of the same name in the corresponding build directory to stop the target from always being considered out-of-date. Alternatively, if your configure script(s) generate output files, you could list them in this array.

Successful build of Kicad 4.0.6 in Linux Mageia 5 via fixing a wx-3.0 symbol

I have managed to build the Kicad 4.0.6 in Linux Mageia 5.1 with gcc version 4.9.2. I first manually fixed two wxWidgets 3.0.2 header files in the /usr/include/wx-3.0/wx/ directory: regex.h and features.h. Kicad then compiled successfully. With the native wx-3.0 headers, the compiler generated the error in pcbnew/netlist_reader.cpp due to the undefined variable wxRE_ADVANCED.
The features.h header checks if the macro WX_NO_REGEX_ADVANCED is defined. If yes, features.h UNdefines wxHAS_REGEX_ADVANCED macro, and defines it, if no. The macro wxHAS_REGEX_ADVANCED, in turn, is used in regex.h to determine if among the enum constants wxRE_ADVANCED = 1 is present. The standard prebuilt Mageia 5 packages wxgtku3.0_0 and lib64wxgtku3.0-devel that I installed with the use of Mageia's software manager urpmi from Mageia repository WX_NO_REGEX_ADVANCED is defined, therefore wxHAS_REGEX_ADVANCED is undefined, and, hence, wxRE_ADVANCED is undefined either. Kicad 4.0.6 source package assumes wxRE_ADVANCED = 1, therefore the build process stops with the error.
Then I reverted /usr/include/wx-3.0/wx/regex.h and features.h to their original state and learned how to add the definition of wxRE_ADVANCED to CMakeLists.txt. However, I still have a question.
The recommended format of adding the definition to CMakeLists.txt I found at CMake command line for C++ #define is this:
if (NOT DEFINED wxRE_ADVANCED)
set(wxRE_ADVANCED 1)
endif()
add_definitions(-DwxRE_ADVANCED=$(wxRE_ADVANCED))
However, it did not work! The macro expansion for wxRE_ADVANCED in pcbnew/netlist_reader.cpp was empty. I printed it at compile time inserting the following lines into the netlist_reader.cpp file (this was hard to find, most of the recommended formats did not work. The correct one is in C preprocessor: expand macro in a #warning):
#define __STRINGIFY(TEXT) #TEXT
#define __WARNING(TEXT) __STRINGIFY(GCC warning TEXT)
#define WARNING(VALUE) __WARNING(__STRINGIFY(wxRE_ADVANCED = VALUE))
Pragma (WARNING(wxRE_ADVANCED))
Finally, I simplified the CMakeLists.txt definition down to this, and it was a success:
if (NOT DEFINED wxRE_ADVANCED)
set(wxRE_ADVANCED 1)
endif()
add_definitions(-DwxRE_ADVANCED=1)
My question: what is the meaning of "-DwxRE_ADVANCED=$(wxRE_ADVANCED)" if it does not work? Is it possible not to use set(wxRE_ADVANCED 1), and simply write add_definitions(-DwxRE_ADVANCED=1)? Thank you.
P.S. Yes, the Kicad 4.0.6 build process successfully finished with only one line added to the top level CMakeLists.txt file:
add_definitions(-DwxRE_ADVANCED=1)
A variable is called via $variable or ${variable}. Note the curly brackets, not parentheses.
Also, it is recommended to use:
target_compile_definitions(mytarget PUBLIC wxRE_ADVANCED=1)
on a target directly, rather than the general add_definitions() command.

How to use the program's exit status at compile time?

This question is subsequent to my previous one: How to integrate such kind of source generator into CMake build chain?
Currently, the C source file is generated from XS in this way:
set_source_files_properties(${CMAKE_CURRENT_BINARY_DIR}/${file_src_by_xs} PROPERTIES GENERATED 1)
add_custom_target(${file_src_by_xs}
COMMAND ${XSUBPP_EXECUTABLE} ${XSUBPP_EXTRA_OPTIONS} ${lang_args} ${typemap_args} ${file_xs} >${CMAKE_CURRENT_BINARY_DIR}/${file_src_by_xs}
WORKING_DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR}
DEPENDS ${file_xs} ${files_xsh} ${_XSUBPP_TYPEMAP_FILES}
COMMENT "generating source from XS file ${file_xs}"
)
The GENERATED property let cmake don't check the existence of this source file at configure time, and add_custom_target let the xsubpp always re-run at each compile. The reason for always rerun is because xsubpp will generate an incomplete source file even if it fails, so there are possibility that the whole compiling continues with an incomplete source file.
I found it is time consuming to always re-run source generator and recompile it. So I want to have it re-run only when dependent XS files are modified. However, if I do so, the incomplete generated source file must be deleted.
So my question is: is there any way to remove the generated file, only when the program exit abnormally at compile time?
Or more generic: is there any way to run a command depending on another command's exit status at compile time?
You can always write a wrapper script in your favorite language, e.g. Perl or Ruby, that runs xsubpp and deletes the output file if the command failed. That way you can be sure that if it exists, it is correct.
In addition, I would suggest that you use the OUTPUT keyword of add_custom_command to tell CMake that the file is a result of executing the command. (And, if you do that, you don't have to set the GENERATED property manually.)
Inspired by #Lindydancer's answer, I achieved the purpose by multiple COMMANDs in one target, and it don't need to write an external wrapper script.
set(source_file_ok ${source_file}.ok)
add_custom_command(
OUTPUT ${source_file} ${source_file_ok}
DEPENDS ${xs_file} ${xsh_files}
COMMAND rm -f ${source_file_ok}
COMMAND xsubpp ...... >${source_file}
COMMAND touch ${source_file_ok}
)
add_library(${xs_lib} ${source_file})
add_dependencies(${xs_lib} ${source_file} ${source_file_ok})
The custom target has 3 commands. The OK file only exists when xsubpp is success, and this file is added as a dependency of the library. When xsubpp is not success, the dependency on the OK file will force the custom command to be run again.
The only flaw is cross-platform: not all OS have touch and rm, so the name of these two commands should be decided according to OS type.

Why won't MSBuild build a project with a dot in the name?

The Story So Far
I've got a nice solution with a desktop application project, a few library projects, and a couple of development tools projects (also desktop applications). At the moment, my build server outputs all of the code into one OutputPath. So we end up with
drop-x.y.z\
Company.MainApplication.exe <-- main application
Company.MainApplicationCore.dll <-- libraries
Helper.exe <-- developer tools
Grapher.exe
Parser.exe
... <-- the rest of the output
But, we're growing up and people outside of our team want access to our tools. So I want to organize the output. I decided that what we would want is a different OutputPath per executable project
drop-x.y.z\
Company.MainApplication\
Company.MainApplication.exe <-- main application
Company.MainApplicationCore.dll <-- libraries
... <-- application specific output
Helper\
Helper.exe <-- developer tools
... <-- tool specific output
Grapher\
Grapher.exe
...
Parser\
Parser.exe
...
What I Did
I found this simple command. I like it because it retains all the Solution working-dir context that makes msbuild a pain.
msbuild /target:<ProjectName>
For example, from my solution root as a working directory, I would call
PS> msbuild /target:Helper /property:OutputPath="$pwd\out\Helper"
I'm testing this from PowerShell, so that $pwd resolves to the full path to my working directory, or the Solution root in this case. I get the output I desire.
However, when I run this command
PS> msbuild /target:Company.MainApplication /property:OutputPath="$pwd\out\Company.MainApplication"
I get the following error output (there's no more information, I ran with /verbosity:diagnostic)
The target "Company.MainApplication" does not exist in the project.
What I Need
The command fails on any project with a dot or dots in the name. I tried with many combinations of working directories and properties. I tried several ways of escaping the property values. I also tried running the command from a <Task> in a targets file.
I need to know either
A) How to fix this command to work property
B) How to achieve the same output with minimal friction
Try using an underscore as an escape character for the dot in the target parameter, e.g.
msbuild /target:Company_MainApplication /property:OutputPath="$pwd\out\Company.MainApplication"
Specify the target after the -target: switch in the format :. If the project name contains any of the characters %, $, #, ;, ., (, ), or ', replace them with an _ in the specified target name.
https://learn.microsoft.com/en-us/visualstudio/msbuild/how-to-build-specific-targets-in-solutions-by-using-msbuild-exe?view=vs-2019
Dan Nolan's answer and comments are correct. Just want to supplement the Microsoft documentation.
The /targets: switch is to identify a <Target to run in the project file. You need to supply your .csproj file as a an argument that is not prefixed by a /xx option marker.
You might also want to work based on the .sln file. In that case, you still dont specify the project in the .sln to build in this manner. I'll leave you to search up the correct syntax in case that's what you end up doing.