I have a problem with it when i use Powerlevel10k which is a theme for ZSH?
From the Powerlevel10k FAQ:
Q: What do different symbols in Git status mean?
When using Lean, Classic or Rainbow style, Git status may look like this:
feature:master ⇣42⇡42 *42 merge ~42 +42 !42 ?42
Legend:
| Symbol | Meaning | Source |
| --------| ------------------------------------------------------------------| ---------------------------------------------------- |
| feature | current branch; replaced with #tag or #commit if not on a branch | git status |
| master | remote tracking branch; only shown if different from local branch | git rev-parse --abbrev-ref --symbolic-full-name #{u} |
| ⇣42 | this many commits behind the remote | git status |
| ⇡42 | this many commits ahead of the remote | git status |
| *42 | this many stashes | git stash list |
| merge | repository state | git status |
| ~42 | this many merge conflicts | git status |
| +42 | this many staged changes | git status |
| !42 | this many unstaged changes | git status |
| ?42 | this many untracked files | git status |
See also: How do I change the format of Git status?
If you've created a Git repository at the root of your home directory to store dotfiles, you probably want to ignore untracked files in it. You can achieve this by executing the following command:
git -C ~ config status.showuntrackedfiles no
This will have several effects:
git status will be faster.
git status won't list 171 untracked files.
?171 will disappear from your prompt.
You can undo the above command with the following command:
git -C ~ config --unset status.showuntrackedfiles
if you don't want to see Git status in your prompt while in home directory, add this parameter to ~/.p10k.zsh:
# Don't display Git status in prompt for Git repositories whose workdir matches
# this pattern.
typeset -g POWERLEVEL9K_VCS_DISABLED_WORKDIR_PATTERN='~'
If you don't want to see Git status in your prompt at all, remove vcs from POWERLEVEL9K_LEFT_PROMPT_ELEMENTS array in ~/.p10k.zsh.
How the gitlab-ci cache is working on docker runner?
What is /cache directory?
What is cache_dir?
Where and how files matching the "paths" in "cache" gitlab-ci.yml are stored?
Volume mounted to /cache directory is created automatically on gitlab-runner installation and managed by cache_dir setting
more about cache_dir:
https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runners-section
https://gitlab.com/gitlab-org/gitlab-runner/blob/main/docs/executors/docker.md#the-builds-and-cache-storage
If you modify the /cache storage path, you also need to make sure to mark this
directory as persistent by defining it in volumes = ["/my/cache/"] under the
[runners.docker] section in config.toml.
TLDR
/cache dir is different from cache config in gitlab-ci.yml
/cache dir in job container is where the cached files are stored
files matching to cache config in gitlab-ci.yml are copied to /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number> at the end of job
The "Clear Runner Caches" button in your project "Pipelines" page schedules TO NOT extract /cache/$CI_PROJECT_NAMESPACE/$CI_PROJECT_NAME/<cache-key>-<cache-number>/cache.zip to dir specified in cache config in gitlab-ci.yml (Instead of "removing content of /cache folder as I though at first)
P.S.
There is container named gitlab-runner-cache created on machine with gitlab-runner (https://gitlab.com/gitlab-org/gitlab-runner/blob/af343971874198a1923352107409583b78e8aa80/executors/docker/executor_docker.go#L382)
(Seems like) This container is used to create anonymous volume where /cache data is stored. After the anonymous volume is created this container is stopped.
The job containers (meaning container were your tests typically run) mounts this anonymous volume
Proofs
HAVING gitlab-ci.yml
image: srghma/docker-nixos-with-git-crypt
cache:
key: "test00000" # to reset cache - change this key OR clear cache in project settings page
paths:
- .mycache # gitlab allows only cache dirs that are relative to project root OR /cache (created automatically)
testtest:
script:
- nix-env -i tree
- tree --dirsfirst -L 4 /cache
- ls -al ./.mycache || true
- echo "test" > /cache/test
- mkdir -p ./.mycache
- echo "test" > ./.mycache/test
- tree --dirsfirst -L 4 /cache
- ls -al ./.mycache || true
Output:
on first run
Running with gitlab-runner 11.6.0 (f100a208)
on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 675caa7 feat: cache update
From https://gitlab.com/srghma/myproject
675caa7..3d1e223 nix -> origin/nix
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
0 directories, 0 files
$ ls -al ./.mycache || true
$ echo "test" > /cache/test
ls: ./.mycache: No such file or directory
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
`-- test
0 directories, 1 file
$ ls -al ./.mycache || true
total 12
drwxr-xr-x 2 root root 4096 Feb 24 11:44 .
drwxrwxrwx 20 root root 4096 Feb 24 11:44 ..
-rw-r--r-- 1 root root 5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded
on second run
Running with gitlab-runner 11.6.0 (f100a208)
on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-11...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
| `-- myproject
| `-- test00000-11
| `-- cache.zip
`-- test
3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x 2 root root 4096 Feb 24 11:44 .
drwxrwxrwx 20 root root 4096 Feb 24 11:44 ..
-rw-r--r-- 1 root root 5 Feb 24 11:44 test
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
| `-- myproject
| `-- test00000-11
| `-- cache.zip
`-- test
3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x 2 root root 4096 Feb 24 11:44 .
drwxrwxrwx 20 root root 4096 Feb 24 11:44 ..
-rw-r--r-- 1 root root 5 Feb 24 11:44 test
Creating cache test00000-11...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded
after you clear cache by clicking on "Clear Runner Caches" in your project "Pipelines" page
Running with gitlab-runner 11.6.0 (f100a208)
on srghma_gitlab_runner 9b3980da
Using Docker executor with image srghma/docker-nixos-with-git-crypt ...
Pulling docker image srghma/docker-nixos-with-git-crypt ...
Using docker image sha256:ad3491aae178f629df713e0719750cc445b4881702b6b04b7cf325121f0032bf for srghma/docker-nixos-with-git-crypt ...
Running on runner-9b3980da-project-222-concurrent-0 via myrunner.com...
Fetching changes...
Removing .mycache/
HEAD is now at 3d1e223 feat: cache update
Checking out 3d1e2237 as nix...
Skipping Git submodules setup
Checking cache for test00000-12...
No URL provided, cache will be not downloaded from shared cache server. Instead a local version of cache will be extracted.
Successfully extracted cache
$ nix-env -i tree
installing 'tree-1.8.0'
these paths will be fetched (0.03 MiB download, 0.09 MiB unpacked):
/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0
copying path '/nix/store/dhfq0dsg9a0j5ai78bmh5qlrla8wvcxz-tree-1.8.0' from 'https://cache.nixos.org'...
building '/nix/store/dankqr2x4g5igc4w7lw9xqnn7lcy4f7a-user-environment.drv'...
created 233 symlinks in user environment
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
| `-- myproject
| `-- test00000-11
| `-- cache.zip
`-- test
3 directories, 2 files
$ ls -al ./.mycache || true
ls: ./.mycache: No such file or directory
$ echo "test" > /cache/test
$ mkdir -p ./.mycache
$ echo "test" > ./.mycache/test
$ tree --dirsfirst -L 4 /cache
/cache
|-- srghma
| `-- myproject
| `-- test00000-11
| `-- cache.zip
`-- test
3 directories, 2 files
$ ls -al ./.mycache || true
total 12
drwxr-xr-x 2 root root 4096 Feb 24 11:45 .
drwxrwxrwx 20 root root 4096 Feb 24 11:45 ..
-rw-r--r-- 1 root root 5 Feb 24 11:45 test
Creating cache test00000-12...
.mycache: found 2 matching files
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
Job succeeded
I'm wonder if there is anyway to just export files that have changed in specific revision.
e.g : I have branch with three files :
file.php
file.js
file.css
Just file.js has changed in last commit.
How to use export command to just export changed file (file.js) and prevent exporting others.
Is there any Plugin or external 3rdParty ?
Using bzr export you can specify a single directory to export, but not individual files.
As an alternative, you can get the contents of a file at some past revision like this:
bzr cat -r REV path/to/file > file.rREV
You can get the list of changed files at some past revision with the one-liner:
bzr diff -c REV | grep ^===
To wrap it up, here's a complete one-liner that does just what you asked for: export just the modified files of some specific revision REV into a directory called EX:
bzr diff -cREV | grep '^=== modified file ' | sed -e "s/[^']*//" -e "s/'//g" |\
while read fname; do echo $fname; mkdir -p EX/"$(dirname "$fname")";\
bzr cat -rREV "$fname" > EX/"$fname"; done
It loops over the modified files in revision REV, prepares the export directory EX with all parent directories needed to save the file preserving the path, and finally gets the file with bzr cat and writes it at the correct relative path inside EX.
my tree command returns
tmp
`-- t
`-- e
|-- foo.ps
`-- s
|-- bar.ps
`-- t
`-- baz.ps
How can I create archive ps.tar.gz in tmp directory with following structure:
tmp
|-- ps.tar.gz
| |-- foo.ps
| |-- bar.ps
| `-- baz.ps
`-- t
`-- e
|-- foo.ps
`-- s
|-- bar.ps
`-- t
`-- baz.ps
?
for GNU tar you can use the --xform argument. It takes a sed expression and applies it on each file name. So for removing all characters up to the last /, you may use:
tar -c --xform s:^.*/:: your-files
outfile=$(pwd)/ps.tar;find . -type f | while read file; do
tar rf $outfile -C $(dirname $file) $(basename $file)
done
gzip $outfile
I have a complex project where there are many directories that have POM files, but only some of which are sub-modules (possibly transitively) of a particular parent project.
Obviously, Maven knows the list of relevant files because it parses all the <module> tags to find them. But, I only see a list of the <name>s in the [INFO] comments, not the paths to those modules.
Is there a way to have Maven output a list of all the POM files that provided references to projects that are part of the reactor build for a given project?
This is quite simple but it only gets the artifactId, from the root (or parent) module:
mvn --also-make dependency:tree | grep maven-dependency-plugin | awk '{ print $(NF-1) }'
If you want the directories
mvn -q --also-make exec:exec -Dexec.executable="pwd"
The following command prints artifactId's of all sub-modules:
mvn -Dexec.executable='echo' -Dexec.args='${project.artifactId}' exec:exec -q
Example output:
build-tools
aws-sdk-java-pom
core
annotations
utils
http-client-spi
http-client-tests
http-clients
apache-client
test-utils
sdk-core
...
mvn help:evaluate -Dexpression=project.modules
mvn help:evaluate -Dexpression=project.modules[0]
mvn help:evaluate -Dexpression=project.modules[1]
IFS=$'\n'
modules=($(mvn help:evaluate -Dexpression=project.modules | grep -v "^\[" | grep -v "<\/*strings>" | sed 's/<\/*string>//g' | sed 's/[[:space:]]//'))
for module in "${modules[#]}"
do
echo "$module"
done
Here's a way to do this on Linux outside of Maven, by using strace.
$ strace -o opens.txt -f -e open mvn dependency:tree > /dev/null
$ perl -lne 'print $1 if /"(.*pom\.xml)"/' opens.txt
The first line runs mvn dependency:tree under strace, asking strace to output to the file opens.txt all the calls to the open(2) system call, following any forks (because Java is threaded). This file looks something like:
9690 open("/etc/ld.so.cache", O_RDONLY) = 3
9690 open("/lib/libncurses.so.5", O_RDONLY) = 3
9690 open("/lib/libdl.so.2", O_RDONLY) = 3
The second line asks Perl to print any text inside quotes that happens to end in pom.xml. (The -l flag handles printing newlines, the -n wraps the code single quotes in a loop that simply reads any files on the command line, and the -e handles the script itself which uses a regex to find interesting calls to open.)
It'd be nice to have a maven-native way of doing this :-)
The solution I found is quite simple:
mvn -B -f "$pom_file" org.codehaus.mojo:exec-maven-plugin:1.4.0:exec \
-Dexec.executable=/usr/bin/echo \
-Dexec.args='${basedir}/pom.xml'| \
grep -v '\['
This is a little bit tricky due to the need to grep out the [INFO|WARNING|ERROR] lines and make it usable for scripting but saved me a lot of time since you can put any expression there.
Get exactly name. Not ID. Result is appropriate for mvn -pl.
mvn help:evaluate -Dexpression=project.modules -q -DforceStdout | tail -n +2 | head -n -1 | sed 's/\s*<.*>\(.*\)<.*>/\1/'
or with main pom.xml
cat pom.xml | grep "<module>" | sed 's/\s*<.*>\(.*\)<.*>/\1/'
I don't have a direct answer to the question. But using some kind of "module path" as naming convention for the <name> of my modules works for me. As you'll see, this convention is self explaining.
Given the following project structure:
.
├── pom
│ ├── pom.xml
│ └── release.properties
├── pom.xml
├── samples
│ ├── ejb-cargo-sample
│ │ ├── functests
│ │ │ ├── pom.xml
│ │ │ └── src
│ │ ├── pom.xml
│ │ └── services
│ │ ├── pom.xml
│ │ └── src
│ └── pom.xml
└── tools
├── pom.xml
└── verification-resources
├── pom.xml
└── src
Here is the output of a reactor build:
$ mvn compile
[INFO] Scanning for projects...
[INFO] Reactor build order:
[INFO] Personal Sandbox - Samples - Parent POM
[INFO] Personal Sandbox - Samples - EJB3 and Cargo Sample
[INFO] Personal Sandbox - Tools - Parent POM
[INFO] Personal Sandbox - Tools - Shared Verification Resources
[INFO] Personal Sandbox - Samples - EJB3 and Cargo Sample - Services
[INFO] Personal Sandbox - Samples - EJB3 and Cargo Sample - Functests
[INFO] Sandbox Externals POM
...
This gives IMHO a very decent overview of what is happening, scales correctly, and it's pretty easy to find any module in the file system in case of problems.
Not sure this does answer all your needs though.
I had the same problem but solved it without strace. The mvn exec:exec plugin is used to touch pom.xml in every project, and then find the recently modified pom.xml files:
ctimeref=`mktemp`
mvn --quiet exec:exec -Dexec.executable=/usr/bin/touch -Dexec.args=pom.xml
find . -mindepth 2 -type f -name pom.xml -cnewer "$ctimeref" > maven_projects_list.txt
rm "$ctimeref"
And you have your projects list in the maven_projects_list.txt file.
This is the command I use for listing all pom.xml files inside a project at the root of the project.
find -name pom.xml | grep -v target | sort
What the command do :
find -name pom.xml what I search
grep -v target avoid to list pom.xml inside target/ directory
sort list the result in alphabetical order
An example to list all modules and the parent of each
export REPO_DIR=$(pwd)
export REPO_NAME=$(basename ${REPO_DIR})
echo "${REPO_DIR} ==> ${REPO_NAME}"
mvn exec:exec -q \
-Dexec.executable='echo' \
-Dexec.args='${basedir}:${project.parent.groupId}:${project.parent.artifactId}:${project.parent.version}:${project.groupId}:${project.artifactId}:${project.version}:${project.packaging}' \
| perl -pe "s/^${REPO_DIR//\//\\\/}/${REPO_NAME}/g" \
| perl -pe 's/:/\t/g;'
I prepared the script below as mvn exec:exec runs slow on gitlab. I couldn't find a free time to investigate it more but I'm suspicious about it tries to get a new runner as it needs a new Runtime. So, if you're working with quite limited runners, it affects the overall build time in an unpredictable way if you used mvn exec:exec to determine the modules.
The below snippet gives you the module name, packaging and path to the module
#!/bin/bash
set -e;
mvnOptions='--add-opens java.base/java.lang=ALL-UNNAMED';
string=$(MAVEN_OPTS="$mvnOptions" mvn help:active-profiles)
delimiter='Active Profiles for Project*';
modules=()
while read -r line; do
if [[ $line == $delimiter ]]; then
module=$(echo $line | sed -E "s/.*'(.*):(.*):(.*):(.*)'.*/\2/");
packaging=$(echo $line | sed -E "s/.*'(.*):(.*):(.*):(.*)'.*/\3/");
path=$(MAVEN_OPTS="$mvnOptions" mvn help:evaluate -Dexpression=project.basedir -pl "$module" -q -DforceStdout || true);
if [[ $path == *" $module "* ]]; then
path=$(pwd);
fi
modules+=("$module" "$packaging" "$path")
fi;
done <<< "$string"
size="$(echo ${#modules[#]})";
moduleCount=$(( $size / 3 ));
# prints the found modules
if [ $moduleCount -gt 0 ]; then
echo "$moduleCount module(s) found"
for (( i=0; i<$moduleCount; ++i)); do
line=$(($i + 1));
moduleIndex=$(($i * 3));
pathIndex=$(($i * 3+2));
module=${modules[moduleIndex]};
path=${modules[pathIndex]};
echo " $line. '$module' at '$path'";
done;
fi;