CMake's add_custom_command does not work as advertised - cmake

Question

I cannot, for the life of me, understand why add_custom_command is not working the way it is described in the documentation. I have a json that that I need minimized and converted to a C header that I can include, so I have python script to handle that. The command I have should be dead simple:
add_custom_command(
OUTPUT include/messages.h
COMMAND ${PYTHON_PREFER} ../scripts/json2c.py -i data/messages.json -o include/messages.h -n messages
DEPENDS data/messages.json
WORKING_DIRECTORY ${CMAKE_CURRENT_LIST_DIR})
But as near as I can tell, dependencies simply don't work. I've tried multiple ways to tie this into my main application with the target app.
target_sources(app PRIVATE include/messages.h)
and
add_custom_target(messages_h DEPENDS include/messages.h)
add_dependencies(app messages_h)
do exactly the same thing. (Not using either will never update the file) The header is regenerated every time, so it's not detecting that the input json file hasn't changed. And, the target is only regenerated every OTHER time, so cmake isn't detecting that the header is changing, even though it's the first thing to be updated.
[main] Building folder: CCM
[build] Starting build
[proc] Executing command: C:\Programs\CMake\bin\cmake.EXE --build c:/Users/abram/Workspaces/CCM/build --config Debug --target all --
[build] [1/1 100% :: 0.145] Generating include/messages.h
[build] Build finished with exit code 0
Followed By:
[main] Building folder: CCM
[build] Starting build
[proc] Executing command: C:\Programs\CMake\bin\cmake.EXE --build c:/Users/abram/Workspaces/CCM/build --config Debug --target all --
[build] [1/12 8% :: 0.121] Generating include/messages.h
[build] [2/12 16% :: 0.827] Building CXX object CMakeFiles/app.dir/src/broadcaster.cpp.obj
[build] [3/12 25% :: 0.888] Linking CXX static library app\libapp.a
[build] [4/12 33% :: 1.048] Linking CXX executable zephyr\zephyr_pre0.elf
[build]
[build] [5/12 41% :: 1.266] Generating dev_handles.c
[build] [6/12 50% :: 1.399] Building C object zephyr/CMakeFiles/zephyr_pre1.dir/dev_handles.c.obj
[build] [7/12 58% :: 1.533] Linking CXX executable zephyr\zephyr_pre1.elf
[build]
[build] [8/12 66% :: 1.610] Generating linker.cmd
[build] [9/12 75% :: 1.773] Generating isr_tables.c, isrList.bin
[build] [11/12 83% :: 1.893] Building C object zephyr/CMakeFiles/zephyr_final.dir/isr_tables.c.obj
[build] [11/12 91% :: 1.912] Building C object zephyr/CMakeFiles/zephyr_final.dir/dev_handles.c.obj
[build] [12/12 100% :: 2.279] Linking CXX executable zephyr\zephyr.elf
[build] Memory region Used Size Region Size %age Used
[build] FLASH: 44672 B 512 KB 8.52%
[build] SRAM: 34432 B 128 KB 26.27%
[build] BACKUP_SRAM: 0 GB 4 KB 0.00%
[build] IDT_LIST: 0 GB 2 KB 0.00%
[build] Build finished with exit code 0
Repeat Ad Infinitum
I am very confused since I find it hard to believe that CMake's dependency system simply doesn't work, or that this function doesn't work as intended. But as far as I understand it according to the documentation, this should work. What gives?
I'm gonna try and convert them to a c file instead of a header which I think CMake will be happier about, but I still think this should have worked.
For further context, I'm using CMake 3.24.0-rc4 Ninja 1.11.0, and GCC 11.2.1 for ARM on Windows 11. I'm using Zephyr RTOS as well which has some heavy CMake frameworking. This could be affecting the behavior, but feel like that would be highly unusual.

Answer 1

The OUTPUT argument to add_custom_command is relative to the current binary directory, so you're creating the wrong file.
You need to adjust your paths to keep everything predictably isolated to the binary directory.
Here's a correct example:
cmake_minimum_required(VERSION 3.23)
project(example)
find_package(Python3 REQUIRED)
add_custom_command(
OUTPUT include/messages.h
COMMAND Python3::Interpreter ../scripts/json2c.py -i data/messages.json -o "${CMAKE_CURRENT_BINARY_DIR}/include/messages.h" -n messages
DEPENDS data/messages.json ../scripts/json2c.py
WORKING_DIRECTORY "${CMAKE_CURRENT_SOURCE_DIR}"
VERBATIM
)
add_executable(app "${CMAKE_CURRENT_BINARY_DIR}/include/messages.h")
target_include_directories(app PRIVATE "${CMAKE_CURRENT_BINARY_DIR}/include")
Don't forget to declare a dependency on the script itself.