qbs cannot find the output of submodule (build with cmake) - cmake

I have a submodule built by cmake and successully build it in qbs script (see below code).
Product {
name: "mylib"
type: ["staticlibrary"]
// define buildScript, outputPath, libPrefix, libExt, cmakeBuildType, cmakeBuildConfig
// Note that outputPath is inside mylib folder, not qbs's build directory
Rule {
multiplex: true
outputArtifacts: [
{
filePath: product.outputPath + product.libPrefix + "lib1static" + product.libExt,
fileTags: ["staticlibrary"]
},
{
filePath: product.outputPath + product.libPrefix + "lib2static" + product.libExt,
fileTags: ["staticlibrary"]
}
]
outputFileTags: ["staticlibrary"]
prepare: {
var cmd = new Command(product.buildScript, [product.outputPath, product.cmakeBuildType, product.cmakeBuildConfig]);
cmd.description = "cmake generate mylib";
cmd.workingDirectory = product.sourceDirectory + "/mylib/build_cmake";
return [cmd];
}
}
However, sometimes qbs delete the "lib1static" and "lib2static" and I get this error while linking:
clang: error: no such file or directory: '<path>/lib1static.a'
clang: error: no such file or directory: '<path>/lib2static.a'
I tried to debug but still don't know why. The first build can be successful with all output libraries are built, but in the next build, all libraries are disappeared.
Can anyone here explain and tell me which is the best practice for this case?

I don't see why the files would get deleted. However, there is another problem:
The rule as shown above will only run once, because it has no inputs, so changes in the source files of your sub-project will not get picked up.
(And, as a side note: If the outputs are not "dynamic", you can declare Artifact items, rather than using the outputArtifacts and outputFileTags properties.)
Try this:
Rule {
multiplex: true
alwaysRun: true
Artifact {
filePath: product.outputPath + product.libPrefix + "lib1static" + product.libExt
fileTags: ["staticlibrary"]
alwaysUpdated: false
}
Artifact {
filePath: product.outputPath + product.libPrefix + "lib2static" + product.libExt
fileTags: ["staticlibrary"]
alwaysUpdated: false
}
prepare: {
var cmd = new Command(product.buildScript, [product.outputPath, product.cmakeBuildType, product.cmakeBuildConfig]);
cmd.description = "cmake generate mylib";
cmd.workingDirectory = product.sourceDirectory + "/mylib/build_cmake";
return [cmd];
}
}

Related

How do I force rebuild of all snapshot dependencies in my teamcity build trigger?

I have a list of build types in my settings.kts:
val buildChain = listOf(build1, build2, build3, ...)
I define the following chain of snapshot dependencies:
for ((previous, current) in buildChain.zip(buildChain.drop(1))) {
current.dependencies {
snapshot(previous) {
reuseBuilds = ReuseBuilds.NO
onDependencyFailure = FailureAction.FAIL_TO_START
onDependencyCancel = FailureAction.FAIL_TO_START
}
}
}
With that, build<N> depends on build<N-1>. Then I define the following build type that runs the whole build chain:
val buildChainRunner = BuildChainRunner(headOfBuildChain = buildChain.last())
where
class BuildChainRunner(
headOfBuildChain: BuildType,
) : BuildType({
name = "Build Chain Runner"
dependencies {
snapshot(headOfBuildChain) {
reuseBuilds = ReuseBuilds.NO
onDependencyFailure = FailureAction.ADD_PROBLEM
onDependencyCancel = FailureAction.ADD_PROBLEM
}
}
triggers {
schedule {
schedulingPolicy = daily {
hour = 2
minute = 0
timezone = "Europe/Amsterdam"
}
branchFilter = "+:<default>"
triggerBuild = always()
withPendingChangesOnly = false
}
}
[...]
}
Now, when the build gets triggered by the schedule trigger, it doesn't necessarily rebuilds all builds from the chain. What I would like to do is the very same as this:
I thought that would be automatically the case because of
reuseBuilds = ReuseBuilds.NO
but unfortunately, it doesn't work. When I trigger the build manually and set "rebuild snapshot dependencies" to "all", then it works as I expect. What am I doing wrong in my kotlin DSL configuration?

Build a CMake project with Nix

I'm trying to make a derivation from fokus (a KDE plasmoid).
I have a default.nix which looks like:
{ mkDerivation, lib, fetchFromGitLab, cmake, extra-cmake-modules
, plasma-framework
, qt5 }:
let
version = "1.5.4";
repo = fetchFromGitLab {
owner = "divinae";
repo = "focus-plasmoid";
rev = "v${version}";
sha256 = "19nndmqfzdcg7lzcifxa17m2mx85dlf4m2wxzr72gf78h4yj0ahv";
};
in
mkDerivation rec {
inherit version;
pname = "fokus";
src = "${repo}/packages";
dontUnpack = true;
buildInputs = [
plasma-framework
qt5.qtbase
];
nativeBuildInputs = [ cmake extra-cmake-modules ];
meta = with lib; {
description = "Simple pomodoro timer plasmoid";
homepage = "https://store.kde.org/p/1308861/";
license = licenses.gpl3Only;
maintainers = with maintainers; [ ];
platforms = platforms.unix;
};
}
I also added to all-packages:
fokus-plasma-applet = libsForQt5.callPackage ../applications/misc/fokus-plasma-applet { };
The thing is, I have only find few examples in nixpkgs, and I do not see what to do with this error (since CMakeLists.txt is effectively in /package):
building '/nix/store/c21h5667abmj4v0ci4n51ckvkxvc23gk-fokus-1.5.4.drv'...
qtPreHook
patching sources
configuring
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/doc/fokus -DCMAKE_INSTALL_INFODIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/include -DCMAKE_INSTALL_SBINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/4g5fidyk19akng9n91cql60ylrjq93s6-gcc-wrapper-10.3.0/bin/strip -DCMAKE_RANLIB=/nix/store/ffcrrgb6k5h7crr93dj90bi9fy1jvj16-binutils-2.35.1/bin/ranlib -DCMAKE_AR=/nix/store/ffcrrgb6k5h7crr93dj90bi9fy1jvj16-binutils-2.35.1/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4 -DKDE_INSTALL_EXECROOTDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4 -DKDE_INSTALL_BINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/bin -DKDE_INSTALL_SBINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/sbin -DKDE_INSTALL_LIBDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib -DKDE_INSTALL_LIBEXECDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/libexec -DKDE_INSTALL_CMAKEPACKAGEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib/cmake -DKDE_INSTALL_INCLUDEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/include -DKDE_INSTALL_LOCALSTATEDIR=/var -DKDE_INSTALL_DATAROOTDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share -DKDE_INSTALL_DATADIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share -DKDE_INSTALL_DOCBUNDLEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/doc/HTML -DKDE_INSTALL_KCFGDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/config.kcfg -DKDE_INSTALL_KCONFUPDATEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/kconf_update -DKDE_INSTALL_KSERVICES5DIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/kservices5 -DKDE_INSTALL_KSERVICETYPES5DIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/kservicetypes5 -DKDE_INSTALL_KXMLGUI5DIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/kxmlgui5 -DKDE_INSTALL_KNOTIFY5RCDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/knotifications5 -DKDE_INSTALL_ICONDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/icons -DKDE_INSTALL_LOCALEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/locale -DKDE_INSTALL_SOUNDDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/sounds -DKDE_INSTALL_TEMPLATEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/templates -DKDE_INSTALL_WALLPAPERDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/wallpapers -DKDE_INSTALL_APPDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/applications -DKDE_INSTALL_DESKTOPDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/desktop-directories -DKDE_INSTALL_MIMEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/mime/packages -DKDE_INSTALL_METAINFODIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/appdata -DKDE_INSTALL_MANDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/man -DKDE_INSTALL_INFODIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/info -DKDE_INSTALL_DBUSDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/dbus-1 -DKDE_INSTALL_DBUSINTERFACEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/dbus-1/interfaces -DKDE_INSTALL_DBUSSERVICEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/dbus-1/services -DKDE_INSTALL_DBUSSYSTEMSERVICEDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/share/dbus-1/system-services -DKDE_INSTALL_SYSCONFDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/etc -DKDE_INSTALL_CONFDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/etc/xdg -DKDE_INSTALL_AUTOSTARTDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/etc/xdg/autostart -DKDE_INSTALL_QTPLUGINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib/qt-5.15.2/plugins -DKDE_INSTALL_PLUGINDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib/qt-5.15.2/plugins -DKDE_INSTALL_QMLDIR=/nix/store/5rdpaqmcj9hrpwdcnprr0zm51659mp08-fokus-1.5.4/lib/qt-5.15.2/qml
CMake Error: The source directory "/build" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
builder for '/nix/store/c21h5667abmj4v0ci4n51ckvkxvc23gk-fokus-1.5.4.drv' failed with exit code 1
cannot build derivation '/nix/store/b9jhnh8bix892wql3qifvgf742lq41v9-system-path.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/3n3mwi7vqmnkng6r8kpa9881xdgcd253-nixos-system-looping-21.11pre328987.c935f5e0add.drv': 1 dependencies couldn't be built
error: build of '/nix/store/3n3mwi7vqmnkng6r8kpa9881xdgcd253-nixos-system-looping-21.11pre328987.c935f5e0add.drv' failed

How can I include kotlin-reflect in the classpath of the Bazel compiler?

I'm trying to get moshi-kotlin-codegen to run on some Kotlin code via Bazel. After a lot of trial and error, I managed to get the plugin to run, but it's failing due to not having kotlin-reflect on the classpath. This is needed by kotlinpoet, which is used by Moshi, so it should be transitively included, AFAICT. However, even explicitly stating the dependency in the BUILD.bazel file for moshi-kotlin-codegen doesn't make it work, so I can only assume it gets filtered out somewhere.
The WORKSPACE file:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "rules_jvm_external",
sha256 = "62133c125bf4109dfd9d2af64830208356ce4ef8b165a6ef15bbff7460b35c3a",
strip_prefix = "rules_jvm_external-3.0",
url = "https://github.com/bazelbuild/rules_jvm_external/archive/3.0.zip",
)
load("#rules_jvm_external//:defs.bzl", "maven_install")
maven_install(
artifacts = [
"com.github.ajalt:clikt:2.6.0",
"org.eclipse.jgit:org.eclipse.jgit:5.7.0.202003090808-r",
"io.github.microutils:kotlin-logging:1.7.8",
"ch.qos.logback:logback-classic:1.2.3",
"com.github.scribejava:scribejava-core:6.9.0",
"com.squareup.moshi:moshi:1.9.2",
"com.squareup.moshi:moshi-kotlin-codegen:1.9.2",
"org.kohsuke:github-api:1.108",
"com.github.ben-manes.caffeine:caffeine:2.8.2",
"javax.xml.bind:jaxb-api:2.3.1",
"org.junit.jupiter:junit-jupiter:5.6.0",
"org.junit.jupiter:junit-jupiter-params:5.6.0",
"com.google.truth:truth:1.0.1",
],
fetch_sources = True,
repositories = [
"https://maven.google.com",
"https://repo1.maven.org/maven2",
"https://jcenter.bintray.com/",
],
strict_visibility = True,
)
rules_kotlin_version = "legacy-1.4.0-rc3"
rules_kotlin_sha = "da0e6e1543fcc79e93d4d93c3333378f3bd5d29e82c1bc2518de0dbe048e6598"
http_archive(
name = "io_bazel_rules_kotlin",
urls = ["https://github.com/bazelbuild/rules_kotlin/releases/download/%s/rules_kotlin_release.tgz" % rules_kotlin_version],
sha256 = rules_kotlin_sha,
)
load("#io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kotlin_repositories", "kt_register_toolchains")
kotlin_repositories()
kt_register_toolchains()
The BUILD.bazel for moshi-kotlin-codegen:
java_plugin(
name = "moshi_kotlin_codegen_plugin",
processor_class = "com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor",
deps = [
"#maven//:com_squareup_moshi_moshi_kotlin_codegen",
],
generates_api = True,
visibility = ["//visibility:public"],
)
(I also tried adding a java_library and depending on that, no luck.)
The final BUILD file that tries to include it:
load("#io_bazel_rules_kotlin//kotlin:kotlin.bzl", "kt_jvm_binary")
kt_jvm_binary(
name = "myproject",
srcs = glob([
"**/*.kt",
]),
main_class = "my.project.MainKt",
plugins = [
"//third_party/moshi_kotlin_codegen:moshi_kotlin_codegen_plugin",
],
deps = [
"#maven//:ch_qos_logback_logback_classic",
"#maven//:com_github_ajalt_clikt",
"#maven//:com_github_ben_manes_caffeine_caffeine",
"#maven//:com_github_scribejava_scribejava_core",
"#maven//:com_squareup_moshi_moshi",
"#maven//:io_github_microutils_kotlin_logging",
"#maven//:org_eclipse_jgit_org_eclipse_jgit",
"#maven//:org_kohsuke_github_api",
"#maven//:javax_xml_bind_jaxb_api",
],
)
The exception during the compilation:
Caused by: kotlin.jvm.KotlinReflectionNotSupportedError: Kotlin reflection implementation is not found at runtime. Make sure you have kotlin-reflect.jar in the classpath
at kotlin.jvm.internal.ClassReference.error(ClassReference.kt:79)
at kotlin.jvm.internal.ClassReference.getQualifiedName(ClassReference.kt:15)
at com.squareup.kotlinpoet.ClassNames.get(ClassName.kt:49)
at com.squareup.moshi.kotlinpoet.classinspector.elements.ElementsClassInspector.<clinit>(ElementsClassInspector.kt:493)
at com.squareup.moshi.kotlin.codegen.JsonClassCodegenProcessor.process(JsonClassCodegenProcessor.kt:99)
at org.jetbrains.kotlin.kapt3.base.incremental.IncrementalProcessor.process(incrementalProcessors.kt)
at org.jetbrains.kotlin.kapt3.base.ProcessorWrapper.process(annotationProcessing.kt:147)
at jdk.compiler/com.sun.tools.javac.processing.JavacProcessingEnvironment.callProcessor(JavacProcessingEnvironment.java:980)
... 48 more
Turns out that this was indeed a bug. A fix is https://github.com/bazelbuild/rules_kotlin/pull/354.

How can I override the jetbrains jdk dependency required by idea-ultimate or idea-community in config.nix?

I’ve been setting up my local nix config as per nixpkgs manual's declarative package management.
I’d like to include idea-ultimate as one of myPackages, but at this time the dependency idea has on the jetbrains jdk is broken, pointing to a non-existing package for macOS.
It’s trying to download jbrsdk-11_0_2-osx-x64-b485.1.tar.gz instead of jbrsdk-11_0_4-osx-x64-b485.1.tar.gz.
I was assuming I could fix that by overriding jetbrainsjdk as follows, but I’m getting: error: attribute 'jetbrainsjdk' missing, at /Users/ldeck/.config/nixpkgs/config.nix:4:20 when I do anything like nix-env -qa ‘jetbrains.*’.
What is the right way to override idea-ultimate so that it uses the fixed jdk?
Here’s my ~./config/nixpkgs/config.nix.
{
allowUnfree = true;
packageOverrides = pkgs: rec {
jetbrainsjdk = pkgs.jetbrainsjdk.override {
version = "520.11";
src = pkgs.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b${jetbrainsjdk.version}.tar.gz";
sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
};
};
myProfile = pkgs.writeText "my-profile" ''
export PATH=$HOME/.nix-profile/bin:/nix/var/nix/profiles/default/bin:/sbin:/bin:/usr/sbin:/usr/bin
export MANPATH=$HOME/.nix-profile/share/man:/nix/var/nix/profiles/default/share/man:/usr/share/man
'';
myPackages = with pkgs; buildEnv {
name = "my-packages";
paths = [
(runCommand "profile" {} ''
mkdir -p $out/etc/profile.d
cp ${myProfile} $out/etc/profile.d/my-profile.sh
'')
aspell
bc
coreutils
direnv
emacs
emscripten
ffmpeg
gdb
git
hello
jq
nixops
nox
scala
silver-searcher
];
pathsToLink = [ "/share/man" "/share/doc" "/bin" "/etc" "/Applications" ];
extraOutputsToInstall = [ "man" "doc" ];
};
};
}
UPDATE 1
Thanks to #ChrisStryczynski who suggested I needed with pkgs, I’ve gotten a little further.
But now the problem is when attempting to install idea-ultimate with the custom jdk, it’s still requiring the broken, non-existing, jbrsdk-11_02-osx-x64-b485.1.tar.gz.drv from somewhere.
Updated config and logs below.
{
allowUnfree = true;
packageOverrides = pkgs: **with pkgs;** rec {
myJetbrainsJdk = **pkgs.jetbrains.jdk.overrideAttrs** (oldAttrs: rec {
version = "520.11";
src = pkgs.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
sha256 = "0d1qwbssc8ih62rlfxxxcn8i65cjgycdfy1dc1b902j46dqjkq9z";
};
});
myIdeaUltimate = pkgs.jetbrains.idea-ultimate.override {
jdk = myJetbrainsJdk;
};
...
myPackages = with pkgs; buildEnv {
...
myIdeaUltimate
];
...
};
};
}
Logs
nix-channel --update; nix-env -iA nixpkgs.myPackages
unpacking channels...
replacing old 'my-packages'
installing 'my-packages'
these derivations will be built:
/nix/store/9kfi3k9q6hi7z3lwann318hndbah535v-idea-ultimate.desktop.drv
/nix/store/ica1m5yq3f3y05xnw7ln1lnfvp0yjvyf-download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz.drv
/nix/store/bf2hwhrvfl8g77gdiw053rayh06x0120-jetbrainsjdk-520.11.drv
/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv
/nix/store/fwwk976sd278zb68zy9wm5pkxss0rnhg-jetbrainsjdk-485.1.drv
/nix/store/s3m2bpcyrnx9dcq4drh95882n0mk1d6m-ideaIU-2019.2.4-no-jbr.tar.gz.drv
/nix/store/9kiajpmmsp3i6ysj4vdqq8dzi84mnr73-idea-ultimate-2019.2.4.drv
/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv
these paths will be fetched (0.02 MiB download, 0.12 MiB unpacked):
/nix/store/hp90sbwznq1msv327f0lb27imvcvi80h-libnotify-0.7.8
building '/nix/store/9kfi3k9q6hi7z3lwann318hndbah535v-idea-ultimate.desktop.drv'...
copying path '/nix/store/hp90sbwznq1msv327f0lb27imvcvi80h-libnotify-0.7.8' from 'https://cache.nixos.org'...
building '/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv'...
trying https://bintray.com/jetbrains/intellij-jdk/download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0
curl: (22) The requested URL returned error: 404 Not Found
error: cannot download download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz from any mirror
builder for '/nix/store/fazsa1a4l70s391rjk9yyi2hvrg0zbmp-download_file?file_path=jbrsdk-11_0_2-osx-x64-b485.1.tar.gz.drv' failed with exit code 1
building '/nix/store/ica1m5yq3f3y05xnw7ln1lnfvp0yjvyf-download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz.drv'...
cannot build derivation '/nix/store/fwwk976sd278zb68zy9wm5pkxss0rnhg-jetbrainsjdk-485.1.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/9kiajpmmsp3i6ysj4vdqq8dzi84mnr73-idea-ultimate-2019.2.4.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv': 1 dependencies couldn't be built
error: build of '/nix/store/jh1ixm54qinv8pk6kypvv6n6cfr4sws8-my-packages.drv' failed
Thanks to How do you discover the override attributes for a derivation, a solution has been found using an overlay.
# ~/config/nixpkgs/overlays/02-jetbrains.nix
self: super:
{
jetbrains = super.jetbrains // {
jdk = super.jetbrains.jdk.overrideAttrs (oldAttrs: rec {
version = "520.11";
src = super.fetchurl {
url = "https://bintray.com/jetbrains/intellij-jbr/download_file?file_path=jbrsdk-11_0_4-osx-x64-b520.11.tar.gz";
sha256 = "3fe1297133440a9056602d78d7987f9215139165bd7747b3303022a6f5e23834";
};
passthru = oldAttrs.passthru // {
home = "${self.jetbrains.jdk}/Contents/Home";
};
});
idea-ultimate = super.jetbrains.idea-ultimate.overrideAttrs (_: {
name = "idea-ultimate.2019.2.4";
src = super.fetchurl {
url = "https://download.jetbrains.com/idea/ideaIU-2019.2.4-no-jbr.tar.gz";
sha256 = "09mz4dx3zbnqw0vh4iqr8sn2s8mvgr7zvn4k7kqivsiv8f79g90a";
};
});
};
}
Install: nix-env -iA 'nixpkgs.jetbrains.idea-ultimate'.
Execute: idea-ultimate.
The crucial part of the puzzle was overriding the passthru.home variable to point to the overlayed JDK rather than the one required by super. Otherwise, you’ll be downloading the old JDK for runtime purposes.
passthru = oldAttrs.passthru // {
home = "${self.jetbrains.jdk}/Contents/Home”;
};
Without /Contents/Home appended, idea won’t startup since self.jetbrains.jdk isn’t a valid home.
Instead of:
nix-env -iA nixpkgs.myPackages
Just do:
nix-env -iA nixpkgs.myIdeaUltimate
The problem being:
myPackages = with pkgs; buildEnv {
...
myIdeaUltimate
];
...
};
Here you are still referencing the old pkgs.myIdeaUltimate.
Nixos seems to do some processing that replaces the pkgs with the appropriate from packageOverrides.

Gradle: How to Display Test Results in the Console in Real Time?

I would like to see test results ( system.out/err, log messages from components being tested ) as they run in the same console I run:
gradle test
And not wait until tests are done to look at the test reports ( that are only generated when tests are completed, so I can't "tail -f" anything while tests are running )
Here is my fancy version:
import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent
tasks.withType(Test) {
testLogging {
// set options for log level LIFECYCLE
events TestLogEvent.FAILED,
TestLogEvent.PASSED,
TestLogEvent.SKIPPED,
TestLogEvent.STANDARD_OUT
exceptionFormat TestExceptionFormat.FULL
showExceptions true
showCauses true
showStackTraces true
// set options for log level DEBUG and INFO
debug {
events TestLogEvent.STARTED,
TestLogEvent.FAILED,
TestLogEvent.PASSED,
TestLogEvent.SKIPPED,
TestLogEvent.STANDARD_ERROR,
TestLogEvent.STANDARD_OUT
exceptionFormat TestExceptionFormat.FULL
}
info.events = debug.events
info.exceptionFormat = debug.exceptionFormat
afterSuite { desc, result ->
if (!desc.parent) { // will match the outermost suite
def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)"
def startItem = '| ', endItem = ' |'
def repeatLength = startItem.length() + output.length() + endItem.length()
println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
}
}
}
}
You could run Gradle with INFO logging level on the command line. It'll show you the result of each test while they are running. Downside is that you will get far more output for other tasks also.
gradle test -i
Disclaimer: I am the developer of the Gradle Test Logger Plugin.
You can simply use the Gradle Test Logger Plugin to print beautiful logs on the console. The plugin uses sensible defaults to satisfy most users with little or no configuration but also offers a number of themes and configuration options to suit everyone.
Examples
Standard theme
Mocha theme
Usage
plugins {
id 'com.adarshr.test-logger' version '<version>'
}
Make sure you always get the latest version from Gradle Central.
Configuration
You don't need any configuration at all. However, the plugin offers a few options. This can be done as follows (default values shown):
testlogger {
// pick a theme - mocha, standard, plain, mocha-parallel, standard-parallel or plain-parallel
theme 'standard'
// set to false to disable detailed failure logs
showExceptions true
// set to false to hide stack traces
showStackTraces true
// set to true to remove any filtering applied to stack traces
showFullStackTraces false
// set to false to hide exception causes
showCauses true
// set threshold in milliseconds to highlight slow tests
slowThreshold 2000
// displays a breakdown of passes, failures and skips along with total duration
showSummary true
// set to true to see simple class names
showSimpleNames false
// set to false to hide passed tests
showPassed true
// set to false to hide skipped tests
showSkipped true
// set to false to hide failed tests
showFailed true
// enable to see standard out and error streams inline with the test results
showStandardStreams false
// set to false to hide passed standard out and error streams
showPassedStandardStreams true
// set to false to hide skipped standard out and error streams
showSkippedStandardStreams true
// set to false to hide failed standard out and error streams
showFailedStandardStreams true
}
I hope you will enjoy using it.
You can add a Groovy closure inside your build.gradle file that does the logging for you:
test {
afterTest { desc, result ->
logger.quiet "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
}
On your console it then reads like this:
:compileJava UP-TO-DATE
:compileGroovy
:processResources
:classes
:jar
:assemble
:compileTestJava
:compileTestGroovy
:processTestResources
:testClasses
:test
Executing test maturesShouldBeCharged11DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test studentsShouldBeCharged8DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test seniorsShouldBeCharged6DollarsForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
Executing test childrenShouldBeCharged5DollarsAnd50CentForDefaultMovie [movietickets.MovieTicketsTests] with result: SUCCESS
:check
:build
Since version 1.1 Gradle supports much more options to log test output. With those options at hand you can achieve a similar output with the following configuration:
test {
testLogging {
events "passed", "skipped", "failed"
}
}
As stefanglase answered:
adding the following code to your build.gradle (since version 1.1) works fine for output on passed, skipped and failed tests.
test {
testLogging {
events "passed", "skipped", "failed", "standardOut", "standardError"
}
}
What I want to say additionally (I found out this is a problem for starters) is that the gradle test command executes the test only one time per change.
So if you are running it the second time there will be no output on test results. You can also see this in the building output: gradle then says UP-TO-DATE on tests. So its not executed a n-th time.
Smart gradle!
If you want to force the test cases to run, use gradle cleanTest test.
This is slightly off topic but I hope it will help some newbies.
edit
As sparc_spread stated in the comments:
If you want to force gradle to always run fresh tests (which might not always be a good idea) you can add outputs.upToDateWhen {false} to testLogging { [...] }. Continue reading here.
Peace.
Add this to build.gradle to stop gradle from swallowing stdout and stderr.
test {
testLogging.showStandardStreams = true
}
It's documented here.
'test' task does not work for Android plugin, for Android plugin use the following:
// Test Logging
tasks.withType(Test) {
testLogging {
events "started", "passed", "skipped", "failed"
}
}
See the following: https://stackoverflow.com/a/31665341/3521637
As a follow up to Shubham's great answer I like to suggest using enum values instead of strings. Please take a look at the documentation of the TestLogging class.
import org.gradle.api.tasks.testing.logging.TestExceptionFormat
import org.gradle.api.tasks.testing.logging.TestLogEvent
tasks.withType(Test) {
testLogging {
events TestLogEvent.FAILED,
TestLogEvent.PASSED,
TestLogEvent.SKIPPED,
TestLogEvent.STANDARD_ERROR,
TestLogEvent.STANDARD_OUT
exceptionFormat TestExceptionFormat.FULL
showCauses true
showExceptions true
showStackTraces true
}
}
My favourite minimalistic version based on Shubham Chaudhary answer.
Put this in build.gradle file:
test {
afterSuite { desc, result ->
if (!desc.parent)
println("${result.resultType} " +
"(${result.testCount} tests, " +
"${result.successfulTestCount} successes, " +
"${result.failedTestCount} failures, " +
"${result.skippedTestCount} skipped)")
}
}
In Gradle using Android plugin:
gradle.projectsEvaluated {
tasks.withType(Test) { task ->
task.afterTest { desc, result ->
println "Executing test ${desc.name} [${desc.className}] with result: ${result.resultType}"
}
}
}
Then the output will be:
Executing test testConversionMinutes [org.example.app.test.DurationTest] with result: SUCCESS
If you have a build.gradle.kts written in Kotlin DSL you can print test results with (I was developing a kotlin multi-platform project, with no "java" plugin applied):
tasks.withType<AbstractTestTask> {
afterSuite(KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
if (desc.parent == null) { // will match the outermost suite
println("Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)")
}
}))
}
Just add the following closure to your build.gradle. the output will be printed after the execution of every test.
test{
useJUnitPlatform()
afterTest { desc, result ->
def output = "Class name: ${desc.className}, Test name: ${desc.name}, (Test status: ${result.resultType})"
println( '\n' + output)
}
}
Merge of Shubham's great answer and JJD use enum instead of string
tasks.withType(Test) {
testLogging {
// set options for log level LIFECYCLE
events TestLogEvent.PASSED,
TestLogEvent.SKIPPED, TestLogEvent.FAILED, TestLogEvent.STANDARD_OUT
showExceptions true
exceptionFormat TestExceptionFormat.FULL
showCauses true
showStackTraces true
// set options for log level DEBUG and INFO
debug {
events TestLogEvent.STARTED, TestLogEvent.PASSED, TestLogEvent.SKIPPED, TestLogEvent.FAILED, TestLogEvent.STANDARD_OUT, TestLogEvent.STANDARD_ERROR
exceptionFormat TestExceptionFormat.FULL
}
info.events = debug.events
info.exceptionFormat = debug.exceptionFormat
afterSuite { desc, result ->
if (!desc.parent) { // will match the outermost suite
def output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} successes, ${result.failedTestCount} failures, ${result.skippedTestCount} skipped)"
def startItem = '| ', endItem = ' |'
def repeatLength = startItem.length() + output.length() + endItem.length()
println('\n' + ('-' * repeatLength) + '\n' + startItem + output + endItem + '\n' + ('-' * repeatLength))
}
}
}
}
Following on from Benjamin Muschko's answer (19 March 2011), you can use the -i flag along with grep, to filter out 1000s of unwanted lines. Examples:
Strong filter - Only display each unit test name and test result and the overall build status. Setup errors or exceptions are not displayed.
./gradlew test -i | grep -E " > |BUILD"
Soft filter - Display each unit test name and test result, as well as setup errors/exceptions. But it will also include some irrelevant info:
./gradlew test -i | grep -E -v "^Executing |^Creating |^Parsing |^Using |^Merging |^Download |^title=Compiling|^AAPT|^future=|^task=|:app:|V/InstrumentationResultParser:"
Soft filter, Alternative syntax: (search tokens are split into individual strings)
./gradlew test -i | grep -v -e "^Executing " -e "^Creating " -e "^Parsing " -e "^Using " -e "^Merging " -e "^Download " -e "^title=Compiling" -e "^AAPT" -e "^future=" -e "^task=" -e ":app:" -e "V/InstrumentationResultParser:"
Explanation of how it works:
The first command is ./gradlew test -i and "-i" means "Info/Verbose" mode, which prints the result of each test in real-time, but also displays large amounts of unwanted debug lines.
So the output of the first command, ./gradlew test -i, is piped to a second command grep, which will filter out many unwanted lines, based on a regular expression. "-E" enables the regular expression mode for a single string; "-e" enables regular expressions for multiple strings; and "|" in the regex string means "or".
In the strong filter, a unit test name and test result is allowed to display using " > ", and the overall status is allowed with "BUILD".
In the soft filter, the "-v" flag means "not containing" and "^" means "start of line". So it strips out all lines that start with "Executing " or start with "Creating ", etc.
Example for Android instrumentation unit tests, with gradle 5.1:
./gradlew connectedDebugAndroidTest --continue -i | grep -v -e \
"^Transforming " -e "^Skipping " -e "^Cache " -e "^Performance " -e "^Creating " -e \
"^Parsing " -e "^file " -e "ddms: " -e ":app:" -e "V/InstrumentationResultParser:"
Example for Jacoco unit test coverage, with gradle 4.10:
./gradlew createDebugCoverageReport --continue -i | grep -E -v "^Executing |^Creating |^Parsing |^Using |^Merging |^Download |^title=Compiling|^AAPT|^future=|^task=|:app:|V/InstrumentationResultParser:"
For Android, this works nicely:
android {
...
testOptions {
unitTests.all {
testLogging {
outputs.upToDateWhen { false }
events "passed", "failed", "skipped", "standardError"
showCauses true
showExceptions true
}
}
}
}
See Running Android unit / instrumentation tests from the console
For those using Kotlin DSL, you can do:
tasks {
named<Test>("test") {
testLogging.showStandardStreams = true
}
}
A more comprehensive response for those using th Kotlin DSL:
subprojects {
// all the other stuff
// ...
tasks.named<Test>("test") {
useJUnitPlatform()
setupTestLogging()
}
}
fun Test.setupTestLogging() {
testLogging {
events(
org.gradle.api.tasks.testing.logging.TestLogEvent.FAILED,
org.gradle.api.tasks.testing.logging.TestLogEvent.PASSED,
org.gradle.api.tasks.testing.logging.TestLogEvent.SKIPPED,
org.gradle.api.tasks.testing.logging.TestLogEvent.STANDARD_OUT,
)
exceptionFormat = org.gradle.api.tasks.testing.logging.TestExceptionFormat.FULL
showExceptions = true
showCauses = true
showStackTraces = true
addTestListener(object : TestListener {
override fun beforeSuite(suite: TestDescriptor) {}
override fun beforeTest(testDescriptor: TestDescriptor) {}
override fun afterTest(testDescriptor: TestDescriptor, result: TestResult) {}
override fun afterSuite(suite: TestDescriptor, result: TestResult) {
if (suite.parent != null) { // will match the outermost suite
val output = "Results: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)"
val startItem = "| "
val endItem = " |"
val repeatLength = startItem.length + output.length + endItem.length
val messages = """
${(1..repeatLength).joinToString("") { "-" }}
$startItem$output$endItem
${(1..repeatLength).joinToString("") { "-" }}
""".trimIndent()
println(messages)
}
}
})
}
}
This should produce an output close to #odemolliens answers.
If you are using jupiter and none of the answers work, consider verifying it is setup correctly:
test {
useJUnitPlatform()
outputs.upToDateWhen { false }
}
dependencies {
testImplementation 'org.junit.jupiter:junit-jupiter-api:5.7.0'
testRuntimeOnly 'org.junit.jupiter:junit-jupiter-engine:5.7.0'
}
And then try the accepted answers
I've written a test logger for Kotlin DSL. You can put this block on your project scope build.gradle.kts file.
subprojects {
tasks.withType(Test::class.java) {
testLogging {
showCauses = false
showExceptions = false
showStackTraces = false
showStandardStreams = false
val ansiReset = "\u001B[0m"
val ansiGreen = "\u001B[32m"
val ansiRed = "\u001B[31m"
val ansiYellow = "\u001B[33m"
fun getColoredResultType(resultType: ResultType): String {
return when (resultType) {
ResultType.SUCCESS -> "$ansiGreen $resultType $ansiReset"
ResultType.FAILURE -> "$ansiRed $resultType $ansiReset"
ResultType.SKIPPED -> "$ansiYellow $resultType $ansiReset"
}
}
afterTest(
KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
println("${desc.className} | ${desc.displayName} = ${getColoredResultType(result.resultType)}")
})
)
afterSuite(
KotlinClosure2({ desc: TestDescriptor, result: TestResult ->
if (desc.parent == null) {
println("Result: ${result.resultType} (${result.testCount} tests, ${result.successfulTestCount} passed, ${result.failedTestCount} failed, ${result.skippedTestCount} skipped)")
}
})
)
}
}
}