I have some "go" code as part of a larger project in a subdir.
I have a .gitlab-ci.yml and the "Security/SAST.gitlab-ci.yml" template enabled.
In that subdir, 'gosec ./...' runs fine but not under gitlab SAST because it starts in the project root. So my use case is looking for a way to get it to 'cp' that subdir to '/go/src/app' instead of the whole project.
...
$ /analyzer run
[INFO] [Gosec] [2022-08-28T09:25:49Z] ▶ GitLab Gosec analyzer v3.5.5
DEBU] [Gosec] [2022-08-28T09:25:49Z] ▶ /bin/cp -r /builds/group11392/project1 /go/src/app
[INFO] [Gosec] [2022-08-28T09:25:49Z] ▶ Fetching dependencies...
[ERRO] [Gosec] [2022-08-28T09:25:49Z] ▶ /usr/local/go/bin/go get -d ./...
go: go.mod file not found in current directory or any parent directory.
...
Related
I really dont know if this is a simple (must be), common or complex task.
I have a buildspec.yml file in my codebuild project, and i am trying to append the version written in package.json file to the output artifact.
I have already seen a lot of tutorials that teach how to append the date (not really useful to me), and others that tell me to execute a version.sh file with this
echo $(sed -nr 's/^\s*"version": "([0-9]{1,}.[0-9]{1,}.*)",$/\1/p' package.json)
and set it in a variable (it doesn't work).
i'm ending up with a build folder called: "my-project-$(version.sh)"
codebuild environment uses ubuntu and nodejs
Update (solved):
my version.sh file:
#!/usr/bin/env bash
echo $(sed -nr 's/^\s*\"version": "([0-9]{1,}\.[0-9]{1,}.*)",$/\1/p' package.json)
Then, i just found out 2 things:
Allow access to your version.sh file:
git update-index --add --chmod=+x version.sh
Declare a variable in any phase in buildspec, i dit in in build phase (just to make sure repository is already copied in environment)
TAGG=$($CODEBUILD_SRC_DIR/version.sh)
then reference it in artifact versioned name:
artifacts:
files:
- '**/*'
name: workover-frontend-$TAG
As result, my build artifact's name: myproject-1.0.0
In my case this script do not want to fetch data from package.json. On my local machine it working great but on AWS doesn't. I had to use chmod in different way, because i got message that i don't have right permissions. My buildspec:
version: 0.2
env:
variables:
latestTag: ""
phases:
pre_build:
commands:
- "echo sed version"
- sed --version
build:
commands:
- chmod +x version.sh
- latestTag=$($CODEBUILD_SRC_DIR/version.sh)
- "echo $latestTag"
artifacts:
files:
- '**/*'
discard-paths: yes
And results in console:
CodeBuild
I also have to mark that when i paste only for example echo 222 into version.sh file i got right answer in CodeBuild console.
My Gitlab CI/CD job has a Maven-cache defined which does not seem to work. The /cache directory as well as the path directory are empty.
This is my .gitlab-ci-yaml:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
MAVEN_CLI_OPTS: "--batch-mode --errors --fail-at-end --show-version -DinstallAtEnd=true -DdeployAtEnd=true"
.cache-m2: &cache-m2
cache:
- key: "mavenRepo"
paths:
- .m2/repository
test_backend:
stage: test
image: maven:3-openjdk-11
tags:
- docker
<<: *cache-m2
script:
- ls -al .m2/repository || true
- mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
- mvn $MAVEN_CLI_OPTS clean test
This is the job log:
Running with gitlab-runner 13.12.0 (7a6612da)
on gitlab-runner-gitlab-runner-54b9d6b99 Ejwdvsgc
Resolving secrets 00:00
Preparing the "kubernetes" executor 00:00
Using Kubernetes namespace: gitlab-runner
Using Kubernetes executor with image maven:3-openjdk-11 ...
Preparing environment 00:03
WARNING: Pulling GitLab Runner helper image from Docker Hub. Helper image is migrating to registry.gitlab.com, for more information see https://docs.gitlab.com/runner/configuration/advanced-configuration.html#migrate-helper-image-to-registrygitlabcom
Waiting for pod gitlab-runner/runner-ejwdvsgc-project-663-concurrent-2chxfp to be running, status is Pending
Running on runner-ejwdvsgc-project-663-concurrent-2chxfp via gitlab-runner-gitlab-runner-54b9d6b99-w8dbf...
Getting source from Git repository 00:02
Fetching changes with git depth set to 50...
Initialized empty Git repository in /builds/Ejwdvsgc/2/t1/mvp/.git/
Created fresh repository.
Checking out 07e45667 as gitlab-test...
Skipping Git submodules setup
Restoring cache 00:01
No URL provided, cache will not be downloaded from shared cache server. Instead a local version of cache will be extracted.
Checking cache for mavenRepo-4...
Successfully extracted cache
Executing "step_script" stage of the job script
$ ls -al .m2/repository || true
ls: cannot access '.m2/repository': No such file or directory
$ mvn help:evaluate -Dexpression=settings.localRepository -q -DforceStdout
/builds/Ejwdvsgc/2/t1/mvp/.m2/repository
...
...
Saving cache for successful job 00:05
Creating cache mavenRepo-4...
.m2/repository: found 4572 matching files and directories
No URL provided, cache will be not uploaded to shared cache server. Cache will be stored only locally.
Created cache
It looks like a cache was found (Checking cache for mavenRepo-4) and it was extracted (Successfully extracted cache)
but the directories do not exists.
What am I doing wrong?
I am experimenting with Skaffold and IntelliJ to develop directly in Kubernetes, but I am having trouble with maven, when IntelliJ try to initialize the environment following erroer occurs in
Running "bash -c curl --fail --show-error --silent --location --retry 3
https://github.com/AdoptOpenJDK/openjdk11-binaries/releases/download/jdk-11.0.10%2B9/OpenJDK11U-
jdk_x64_linux_hotspot_11.0.10_9.tar.gz | tar xz --directory /layers/google.java.runtime/java --
strip-components=1"
[builder] Done "bash -c curl --fail --show-error --silent --location --retry..." (59.3720683s)
[builder] === Java - Maven (google.java.maven#0.9.0) ===
[builder] Installing Maven v3.6.3
[builder] Running "/layers/google.java.maven/maven/bin/mvn clean package --batch-mode -DskipTests --
quiet"
[builder] [ERROR] [ERROR] Some problems were encountered while processing the POMs:
The problem is, some of my Spring Boot Application dependencies are defined in our Nexus Repository and that is defined mirror in my maven settings.xml and this process does not know that mirror configuration and I can't find a way to configure that for skaffold.
I try set settings.xml in skaffold.yml as following
apiVersion: skaffold/v2beta11
kind: Config
build:
artifacts:
- image: myproject/myapp
jib:
args:
- --settings=C:\maven\conf\settings.xml
tagPolicy:
sha256: {}
Anybody had any idea how to let 'google.java.maven' to use my mirror configuration?
Thx for answers...
Skaffold supports three builders work out of the box for Java apps: Jib, Buildpacks, and Docker. The Jib builder will be easiest for your needs.
Jib builds run on your host machine (vs within a containerized environment). Because Skaffold's Jib builder just invokes Maven or Gradle directly, they use your account settings with no additional configuration required (specifically your $HOME/.m2/settings.xml and your artifact cache in $HOME/.m2/repository). Your skaffold.yaml above just needs a small indentation tweak and it should all work:
apiVersion: skaffold/v2beta11
kind: Config
build:
artifacts:
- image: myproject/myapp
jib: {}
tagPolicy:
sha256: {}
You can see a working example in the Skaffold examples.
Docker and Buildpacks builds are run within a container: that is, the source is copied into the container. As a result, you can't reference files outside of the build context, like your $HOME/.m2/settings.xml. You could create a model settings.xml within your source directory and reference that file, and then use environment variables or build-arguments to pass in usernames and passwords. But it becomes quite involved.
We have an open issue to allow mounting directories as volumes for the Buildpacks builder, and we should be able to do the same for the Docker builder. That functionality would make it easier to support your situation if you really wanted to use Buildpacks or Docker.
When I run mvn javadoc:javadoc locally, it gives me a bunch of warnings—empty #return tags, unknown tags etc.—but eventually builds the Javadoc tree.
On GitLab CI, I have the following in .gitlab-ci-yml:
variables:
MAVEN_OPTS: "-Dhttps.protocols=TLSv1.2 -Dmaven.repo.local=$CI_PROJECT_DIR/.m2/repository -Dorg.slf4j.simpleLogger.log.org.apache.maven.cli.transfer.Slf4jMavenTransferListener=WARN -Dorg.slf4j.simpleLogger.showDateTime=true -Djava.awt.headless=true"
# [...]
deploy:jdk8:
stage: deploy
script:
- if [ ! -f ci_settings.xml ];
then echo "CI settings missing\! If deploying to GitLab Maven Repository, please see https://docs.gitlab.com/ee/user/project/packages/maven_repository.html#creating-maven-packages-with-gitlab-cicd for instructions.";
fi
- 'mvn $MAVEN_CLI_OPTS deploy -s ci_settings.xml'
- 'mvn javadoc:javadoc'
- 'cp -r target/site/apidocs public/javadoc/dev'
artifacts:
paths:
- public
only:
- master
- dev
Here, Javadoc generation fails, with the previously mentioned warnings being reported as errors.
I am ultimately going to fix these things, but in the meantime, I would like Javadoc on CI to behave like its local counterpart. Where is the setting to accomplish that?
Changing the mvn javadoc:javadoc line as follows did the trick for me:
- 'mvn javadoc:javadoc -DadditionalJOption=-Xdoclint:none'
Still not sure why this sems to be the default behavior on my local Maven installation but not on GitLab, but at least this solves my issue for now.
Back in Xcode 9, there was a build option called "Clean Build Folder..." (⌥⇧⌘K), which deleted all files in the build folder, only leaving the folder behind with no contents. Since then, this behavior was removed, the menu item's title changed to "Clean Build Folder", and now behaving like the old "Clean" used to.
xcodebuild has a build option called clean which simply does the same thing as Xcode's "Clean Build Folder" (⌘⇧K), which leaves stuff around.
Is there any way to delete all files in the build folder via a scriptable command?
What I've tried so far:
xcodebuild clean -workspace "My Workspace.xcworkspace" -scheme "My Scheme"
This, as I said, doesn't actually clean everything up. For that, I added this bodge to my build script:
export IS_XCODE_CACHE_FOLDER_PRESENT="`ls -la ~/Library/Developer/ | grep -x "Xcode"`"
if [ 0 -ne "$IS_XCODE_CACHE_FOLDER_PRESENT" ]; then
echo "Xcode cache folder should not be present at build time! Attempting to delete..."
rm -rf "~/Library/Developer/Xcode"
RM_RESULT=$?
if [ 0 -ne "$RM_RESULT" ]; then
echo "FAILED to remove Xcode cache folder!"
exit $RM_RESULT
fi
fi
I faced a similar requirement. So after trying for several hours, I resolved to a custom script instead of using Xcode's run script.
So instead of using Xcode to run the app on the simulator I use my script which in turn first cleans the build folder, then builds the project, then installs and finally launches the app in the simulator.
Here is what I am using as a quick script:
# Delete Build directory
rm -rf ./build/Build
# pod install
pod install
# Build project
xcrun xcodebuild -scheme Example -workspace Example.xcworkspace -configuration Debug -destination 'platform=iOS Simulator,name=iPhone 11 Pro Max,OS=13.1' -derivedDataPath build
# Install App
xcrun simctl install "iPhone 11 Pro Max" ./build/Build/Products/Debug-iphonesimulator/Example.app/
# Launch in Simulator
xcrun simctl launch "iPhone 11 Pro Max" com.ihak.arpatech.Example
Note: See this question I posted to know the issue I was facing.
You can add clean action.
xcodebuild clean build -workspace "My Workspace.xcworkspace" -scheme "My Scheme"
see more in man xcodebuild
action ...
Specify one or more actions to perform. Available actions are:
build Build the target in the build root (SYMROOT). This is the default action, and is used if no action is given.
build-for-testing Build the target and associated tests in the build root (SYMROOT). This will also produce an xctestrun file in the build root. This requires speci-
fying a scheme.
analyze Build and analyze a target or scheme from the build root (SYMROOT). This requires specifying a scheme.
archive Archive a scheme from the build root (SYMROOT). This requires specifying a scheme.
test Test a scheme from the build root (SYMROOT). This requires specifying a scheme and optionally a destination.
test-without-building Test compiled bundles. If a scheme is provided with -scheme then the command finds bundles in the build root (SRCROOT). If an xctestrun file is
provided with -xctestrun then the command finds bundles at paths specified in the xctestrun file.
installsrc Copy the source of the project to the source root (SRCROOT).
install Build the target and install it into the target's installation directory in the distribution root (DSTROOT).
clean Remove build products and intermediate files from the build root (SYMROOT).