I am trying to migrate a project from a custom build script to cmake. The source structure looks roughly like this:
src
|
+-CMakeLists.txt
|
+-generated
| |
| +-CMakeLists.txt
| |
| +-database
| |
| +-...
|
+-main
|
+-CMakeLists.txt
|
+-database
|
+-...
The source files in generated/database get autogenerated. cmake seems to be capable of this (that's not part of the question), but I wonder how I can make it build stuff in the right order. main/database contains the framework that is used in the autogenerated files. However, there are other folders in main that depend on the generated sources. If I structure the top-level CMakeLists.txt like this:
add_subdirectory("generated")
add_subdirectory("main")
I cannot refer to main/database in generated/CMakeLists.txt as dependency.
Overall, I have the impression that cmake forces me to structure my files according to their dependencies, but I want to preserve the current layout - the dependencies in the project are far too complex to map them onto a file system hierarchy.
Should I just avoid add_subdirectory and write everything in the top-level CMakeLists.txt? It seems like this should be possible. Or is there another way to solve this?
Overall, I have the impression that cmake forces me to structure
my files according to their dependencies
If CMake has information that target A depends on target B (for example
target_link_libraries(A B)) B will be build first, then A.
If you use generated sources CMake can't get dependency information
and you need to provide it using add_dependencies.
From documentation:
Adding dependencies with this command can be used to make sure one target
is built before another target.
Related
If I have a similar to the following structure project tree (each subproject is added via add_subdirectory() to the parent project):
CMakeLists.txt (TOP-LEVEL)
|
+ -- subproj1
| |
| + -- CMakeLists.txt (subproj1)
| |
| + -- subsubproj1
| |
| + -- CMakeLists.txt (subsubproj1)
|
+ -- subproj2
|
+ -- CMakeLists.txt (subproj2)
I want to expose a variable set inside subsubproj1 in subproj2. The nature of this variable is irrelevant but in my case it points at ${CMAKE_CURRENT_SOURCE_DIR}/include that is the include directory of subsubproj1, which (in my case) is a library used by subproj2. Currently I am re-assigning the variable in each intermediate CMakeLists.txt between the one (here subproj1) where the variable was assigned a value to the top-level with PARENT_SCOPE enabled:
CMakeLists.txt (subsubproj1)
# Expose MY_VAR to subproj1
set(MY_VAR
"Hello"
PARENT_SCOPE
)
CMakeLists.txt (subproj1)
# Expose MY_VAR to top level project thus making it visible to all
set(MY_VAR
${MY_VAR}
PARENT_SCOPE
)
This can be applied to an arbitrary nested project tree.
My question is what is the common practice of doing what I have described above? I can declare MY_VAR as a top-level variable to begin with but what if for some reason I don't want to make it visible (as in written text) there. In which case is PARENT_SCOPE no longer an option and should be replaced with just a straight declaration of that variable in the top-level CMakeLists.txt?
Targets
The nature of this variable is irrelevant but in my case it points at ${CMAKE_CURRENT_SOURCE_DIR}/include that is the include directory of subsubproj1, which (in my case) is a library used by subproj2.
No, the nature is not irrelevant.
Using variables to communicate include directories in CMake is a horrible anti-pattern from the 2.6.x days. You should not use a hammer to drive in a screw.
Non-IMPORTED projects are always global, so you can link to them safely. In subsubproj1 you would write:
add_library(myproj_subsubproj1 INTERFACE)
add_library(myproj::subsubproj1 ALIAS myproj_subsubproj1)
target_include_directories(
myproj_subsubproj1
INTERFACE
"$<BUILD_INTERFACE:${CMAKE_CURRENT_SOURCE_DIR}/include>"
)
Then in subproj2, you would write:
target_link_libraries(myproj_subproj2 PRIVATE myproj::subsubproj1)
Worse options
The following options are worse because they forego the declarative parts of the CMake language and make your scripts dependent on subproject inclusion order. This is a significant complexity increase that (in my experience) is not warranted in build code.
Nevertheless, here are the imperative tools CMake provides:
1. Using a CACHE variable
The cache is a disk-persisted store for global variables. Setting one in any directory makes the value visible to all directories.
Note that there are a few potential drawbacks to this:
Prior to CMake 3.21, creating a new cache entry would delete a normal variable of the same name, leading to tricky situations where builds could become non-idempotent (bad!). See https://cmake.org/cmake/help/latest/policy/CMP0126.html
The user can overwrite your cache variables at the command line, so you cannot rely on either the defined-ness or the value of the variable when your CMake program starts running.
If you can live with this, then you can write:
# On CMake <3.21, honor normal variables. Can remove
# check if on CMake >=3.21
if (NOT DEFINED MyProj_INADVISABLE_VARIABLE)
set(MyProj_INADVISABLE_VARIABLE
"${CMAKE_CURRENT_SOURCE_DIR}/include>"
CACHE STRING "Doc string...")
# If you want to hint to your users that they should
# not edit this variable, include the following line:
mark_as_advanced(MyProj_WEIRD_VARIABLE)
endif ()
If you do not want to allow your users to override it, then you may consistently use an INTERNAL cache variable:
set(MyProj_INADVISABLE_VARIABLE "..." CACHE INTERNAL "...")
As long as you initialize it to a known value early on, then this will work okay as a global variable, but might incur disk traffic on writes.
2. Directory property
A slightly better approach is to use a custom directory property to communicate a value. In subsubproj1:
set_property(DIRECTORY "." PROPERTY inadvisable_prop "foo")
Then in subproj2:
get_property(value DIRECTORY "../subproj1/subsubproj1"
PROPERTY inadvisable_prop)
Note that it is not an error to get a non-existent property, so be on the lookout for types.
You could also use a GLOBAL property instead of a directory property, but global variables in general are a headache waiting to happen. You might as well set it on the directory to decrease the chances of unintended scoping bugs.
How to exclude a single file from module output? I have seen the field "excluded files" in the module configuration (File -> Project structure -> modules) and tried to enter a pattern like that one:
path\to\file\*Server*
in order to exclude the class
path.to.file.MyServer
but it doesn't work
You should go to Settings | Build, Execution, Deployment | Compiler | Excludes and add your file name to the list.
I have an AndroidStudio project with 'C' files in. I can compile and run as-is.
My native files are in
src/main/jni/aes
src/main/jni/libjpeg
src/main/jni/smuglib
I am trying to move the source to a location external to the Android studio project so that I can use it from several locations/projects to avoid copy/paste/mistake cycle.
I have defined the include path in CMakeLists.txt
include_directories(src/main/jni/aes src/main/jni/libjpeg src/main/jni/smuglib)
And have specified the files in the add_library command
add_library( # Sets the name of the library.
native-lib
# Sets the library as a shared library.
SHARED
# Provides a relative path to your source file(s).
src/main/jni/aes/aes.c
src/main/jni/smuglib/smuglib.c
.... etc
How do I set up a variable to refer to these paths, eg 'src/main/jni/aes' so that I can use it in both the include and in the source list?
I tried variations on
set(aes_src, src/main/jni/aes)
but uses of it as ${aes_src} either in the include path statement or in the source list give me all sorts of arcane errors which I am at a loss to understand.
I will generate some of these and include them if folk think it would help, but I am likely barking up the wrong kettle of fish with this approach.
Is there a better approach?
It is set(VAR_NAME item1 item2 item3). No commas needed.
I had a project that source code including non-class file in package(eg:.sql,.ftl). I want to generate output including them but the artifacts always contains class file. How can I include them?
It's a web application, and below is source structure(java code part):
src
|
main
|
java
|
package1
|
Class1.java
Class2.java
sql1.sql
sql2.sql
template1.ftl
The output only contains compiled class.
I found a solution:
Project Structure->Module:at the bottom, Exclude files:*.java
Project Structure->Artifacts:Output Layout:add 'java' directory contents to classes
I want to create a big Jar file. for which am trying to use SBT ASSEMBLY. I installed sbt-assembly from GitHub and this answer. When I ran sbt assembly, I got this error:
java.lang.RuntimeException: deduplicate: different file contents found in the following:
/home/UserName/.ivy2/cache/org.eclipse.jetty.orbit/javax.servlet/orbits/javax.servlet-2.5.0.v201103041518.jar:javax/servlet/SingleThreadModel.class
/home/UserName/.ivy2/cache/org.mortbay.jetty/servlet-api/jars/servlet-api-2.5-20081211.jar:javax/servlet/SingleThreadModel.class
To solve this, I followed User's README page and this is the code he suggests.
mergeStrategy in assembly <<= (mergeStrategy in assembly) { (old) =>
{
case PathList("org", "apache", xs # _*) => MergeStrategy.last
case PathList("javax", "servlet", xs # _*) => MergeStrategy.last
case PathList("com", "esotericsoftware", xs # _*) => MergeStrategy.last
case PathList("project.clj") => MergeStrategy.last
case PathList("overview.html") => MergeStrategy.last
case x => old(x)
}
}
Even after adding this code, I get the same error mentioned before. Please let me know what I am missing. Any help on this error will be appreciated. Thanks!
Update 2 :
Added the exlusion rule as per the link given,
libraryDependencies ++= Seq("org.apache.spark" %% "spark-core" % "0.8.0-incubating","com.codahale" % "jerkson_2.9.1" % "0.5.0","org.skife.com.typesafe.config" % "typesafe-config" % "0.3.0").map(_.exclude("javax", "servlet"))
Update 3:
I can locate the library that is causing the issue.
| +-org.apache.avro:avro-ipc:1.7.4
| | +-io.netty:netty:3.4.0.Final (evicted by: 3.5.4.Final)
| | +-io.netty:netty:3.5.4.Final
...
...
| | +-org.mortbay.jetty:jetty-util:6.1.26
| | +-org.mortbay.jetty:jetty:6.1.26
| | | +-org.mortbay.jetty:jetty-util:6.1.26
| | | +-org.mortbay.jetty:servlet-api:2.5-20081211
| | |
| | +-org.mortbay.jetty:servlet-api:2.5-20081211
| | +-org.slf4j:slf4j-api:1.7.2
...
...
| +-org.eclipse.jetty:jetty-server:7.6.8.v20121106
| | +-org.eclipse.jetty.orbit:javax.servlet:2.5.0.v201103041518
Update 4 : Fix
So adding MergeStrategy did solve the problem. Even though I had quite a few dependencies, more than 10, adding MergeStrategy for each one of them individually solved the issue.
I think you are trying to treat the symptoms, but your problem actually isn't assembly: Your project has a library twice in two different versions on the class path (javax.servlet).
If these versions are binary compatible (which I don't know), you would be fine by excluding one of the two occurrences in your build file like so. If they incompatible, you will need to unroll your dependency graph (a good way to do this might be the sbt-dependency-graph plugin) and try to find matching versions.
In any case, it might be useful to (at least transitorily) keep the libraries in the project folder. If you add retrieveManaged in ThisBuild := true to your sbt build file, all the libraries will be found in <project-root>/lib_managed. This allows you to see which jars are actually there.
EDIT: Showing the dependency graph:
Add to project/plugins.sbt:
addSbtPlugin("net.virtual-void" % "sbt-dependency-graph" % "0.7.4")
Add to build.sbt:
net.virtualvoid.sbt.graph.Plugin.graphSettings
Then run sbt dependency-graph.
You can also refer This Tech Blog
Creating a single JAR for Spark project using sbt-assembly
This post is about how to create a fat jar for spark streaming project using sbt plugin. sbt-assembly is a sbt plugin to create a fat JAR of sbt project with all of its dependencies.