How to glob an absolute path for files in Bazel - tensorflow

Context: I'm trying to come up with a fix for https://github.com/tensorflow/tensorflow/issues/37861 where header files of an external dependency are manually listed but that list is version specific and hence impossible to keep up to date.
What is happening:
tf_http_archive(name = "com_google_protobuf", system_build_file = clean_dep("//third_party/systemlibs:protobuf.BUILD") ...) is invoked
tf_http_archive is a repository_rule with effectively nothing but ctx.template("BUILD.bazel", ctx.attr.system_build_file, {...}, False)
In the protobuf.BUILD there is a list HEADERS = ["google/protobuf/any.pb.h", ...] which is passed to the hdrs argument of cc_library calls
a genrule apperantly symlinks those headers from $(INCLUDEDIR) into $(#D) (I'm not really familiar with Bazel but IIUC the latter is some internal build directory used later)
As I'm unfamiliar with Bazel in general I'll just assume the list of headers is required and there exists a $(INCLUDEDIR)/google/protobuf folder and is somewhere (else) on the system, e.g. /usr/local/include.
Is there any way to get all *.h and *.inc files in the format (i.e. relative to $(INCLUDEDIR)) via a glob or similar? The Bazel glob function doesn't work for absolute paths, so that can't be used.
I found https://github.com/bazelbuild/bazel/issues/8846 suggesting to use new_local_repository with a build_file and a path set to (in this case) $(INCLUDEDIR) but I don't see how that could be applied to the tf_http_archive (which has some conditions to either download the dependency or just use the system_build_file). This seems to also allow to avoid the symlinking (which I'm highly suspicious of anyway because that folder is added via -iquote but include style is #include <...>, see my comments in https://github.com/tensorflow/tensorflow/issues/37861)
Bonus points for people contributing to the issue or ideas why action_env environment variables seem to be ignored in a native.cc_library call.

Related

CMakeLists using variables to define source/include locations

I have an AndroidStudio project with 'C' files in. I can compile and run as-is.
My native files are in
src/main/jni/aes
src/main/jni/libjpeg
src/main/jni/smuglib
I am trying to move the source to a location external to the Android studio project so that I can use it from several locations/projects to avoid copy/paste/mistake cycle.
I have defined the include path in CMakeLists.txt
include_directories(src/main/jni/aes src/main/jni/libjpeg src/main/jni/smuglib)
And have specified the files in the add_library command
add_library( # Sets the name of the library.
native-lib
# Sets the library as a shared library.
SHARED
# Provides a relative path to your source file(s).
src/main/jni/aes/aes.c
src/main/jni/smuglib/smuglib.c
.... etc
How do I set up a variable to refer to these paths, eg 'src/main/jni/aes' so that I can use it in both the include and in the source list?
I tried variations on
set(aes_src, src/main/jni/aes)
but uses of it as ${aes_src} either in the include path statement or in the source list give me all sorts of arcane errors which I am at a loss to understand.
I will generate some of these and include them if folk think it would help, but I am likely barking up the wrong kettle of fish with this approach.
Is there a better approach?
It is set(VAR_NAME item1 item2 item3). No commas needed.

CMake and header search paths

I'm porting a project to CMake, and struggling to find how to set header search paths (previously set with compiler flags e.g. -I "../../Source").
I currently have:
target_include_directories (jni-bridge PRIVATE
"../../Analysis"
"../../Source"
)
But this does not work. How should I set the paths, and what location are they relative to?
Internally, CMake uses absolute paths as include directories. If relative path is used with target_include_directories, then it is interpreted relative to the current source directory (${CMAKE_CURRENT_SOURCE_DIR}).
The above is true when generator expressions are not used.
Generator expression $<INSTALL_INTERFACE:..> may (and strongly recommended to) use relative path which is interpreted relative to the install prefix.
Generator expression $<BUILD_INTERFACE:...> should use absolute path.
In case someone comes here for a simple and direct code example, referring to the example at https://cmake.org/examples/, append the following to the top-level CMakeLists.txt file:
target_include_directories( helloDemo
path/to/header1.h
path/to/header2.h
)

CMake to find path to a file which its name contains a certain string

So I am writing a CMake module to find some libraries which is being used a lot in our company. These libraries all have different versions and are such a mess here. In a certain library, even the name of header files and binary files don't match. So I am writing a CMake script to handle all the problems of finding each library once and for all. What I have in my mind is to write something like how we find boost packages but include the version of each component as well. Something like this:
find_package(OrgName COMPONENTS librarya-1.4.3 libraryb-2.3.1 libraryc-3.6.0)
So I created a FindOrgName.cmake file and iterated on the requested components, then I processed the string which is passed and gained the library name along with its version information something like this (never mind the difference between include and binary files):
IF(OrgName_FIND_COMPONENTS)
FOREACH(comp ${OrgName_FIND_COMPONENTS})
SET(OrgName_${comp}_FOUND 0)
STRING(FIND ${comp} "-" dashind REVERSE)
STRING(LENGTH ${comp} length)
STRING(SUBSTRING ${comp} 0 ${dashind} name)
MATH(EXPR s "${dashind}+1")
MATH(EXPR l "${length}-${dashind}-1")
STRING(SUBSTRING ${comp} ${s} ${l} version)
SET(OrgName_${name}_INCLUDE_DIR "/usr/local/include/OrgName/${comp}/")
find_library(OrgName_${comp}_LIBRARIES NAMES "${comp}" HINTS "/usr/lib")
IF(OrgName_${comp}_INCLUDE_DIR AND OrgName_${comp}_LIBRARIES)
SET(OrgName_${comp}_FOUND 1)
ENDIF()
IF(NOT OrgName_${comp}_FOUND AND OrgName_FIND_REQUIRED_${comp})
MESSAGE(FATAL_ERROR "OrgName ${comp} not available.")
ENDIF()
SET (OrgName_INCLUDE_DIR ${OrgName_INCLUDE_DIR} ${OrgName_${comp}_INCLUDE_DIR})
SET (OrgName_LIBRARIES ${OrgName_LIBRARIES} ${OrgName_${comp}_LIBRARIES})
ENDFOREACH()
ENDIF()
Now here is the problem, imagine someone didn't enter the version part in components names. I want to search for the versions which he has installed (assume the path to library is always the same) and use the last version it can find, so I have to search for the files which their name contains the library name (${name}). First of all how can I do this? Second, Am I doing things right? I mean is there an easier way to do this task?
OK, after a lot of searching and getting exhausted I finally found a solution. I just need to use the following command:
file(GLOB files "Mylib*")
Which will create a list named files and adds each file that its name matches the pattern "Mylib*" to it. I really don't know why they didn't implement it in find_file command but never mind, I'll put it here in case anyone got confused like me.

CMake find_library matching behavior?

One specifies find_library(<var> name PATHS path1..pathn)
My question is how does find_library() match name to the library file (on Windows and Linux)?
For example, I am unable to have find_library() identify the MagicK and MagicK++ DLL files in the provided Windows binary installation of GraphicsMagicK:
The files is: CORE_RL_magick_.dll
Searching for the queries: magick or CORE_RL_magick does not work.
You might want to take a look at this documentation links:
http://www.cmake.org/cmake/help/v2.8.10/cmake.html#command:find_library
http://www.cmake.org/cmake/help/v2.8.10/cmake.html#variable:CMAKE_FIND_LIBRARY_PREFIXES
http://www.cmake.org/cmake/help/v2.8.10/cmake.html#variable:CMAKE_FIND_LIBRARY_SUFFIXES
find_library may accept one or more library names. Those names get the value of CMAKE_FIND_LIBRARY_PREFIXES prepended and CMAKE_FIND_LIBRARY_SUFFIXES
appended. This two variables should be set for each OS depending on how the librares are prefixed or suffixed there.
In your case I'd write for Windows
SET(CMAKE_FIND_LIBRARY_PREFIXES "")
SET(CMAKE_FIND_LIBRARY_SUFFIXES ".lib" ".dll")
and for Linux
SET(CMAKE_FIND_LIBRARY_PREFIXES "lib")
SET(CMAKE_FIND_LIBRARY_SUFFIXES ".so" ".a")
and then write
find_library(
magick
CORE_RL_magick_ (or NAMES if there are multiple names for the same library on different systems)
PATHS
path1
path2
...
(other options that are specified in documentation and would be usefull to you)
)
EDIT:
CMAKE_FIND_LIBRARY_PREFIXES and CMAKE_FIND_LIBRARY_SUFIXES are set automatically by project() command so calling it first and find_library() after that point is a better solution than setting the variables manually.
Why not use find_file() instead of find_library() if you want to find a .dll.

Cmake script problems find_path

I have put these two lines in my Findglm.cmake file to point to the headers for this header library.
find_path(glm_INCLUDE_DIR NAMES glm.hpp matrix_transform.hpp type_ptr.hpp PATHS
${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm ${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm/gtc
${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm/gtx ${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2glm/core)
set(glm_INCLUDE_DIRS ${glm_INCLUDE_DIR})
However when I generate my Xcode project it says that it cannot locate matrix_transform.hpp and type_ptr.hpp
I have played around with this some more it appears to only find the first argument I am wondering if I am using find path wrong ?
I am using cmake 2.8.8 darwinports.
The find_path() command returns single directory. In your case, it's the first dir, which contains the first file.
If this glm will be always located in your source dir, it would be sufficient to do
include_directories(${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm
${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm/gtc
${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm/gtx
${CMAKE_SOURCE_DIR}/libs/glm-0.9.3.2/glm/core)
The find_path() is used to determine dir somewhere outside of your project.