How can I create a virtual machine (Centos-8) using a gitlab-ci.yml file?
No software needed to build the VM should be installed on the Gitlab runner. Binary software should also not be checked into git.
I can create docker images - for example for the use of Packer - in a prebuild step. This image can be cached in Artifactory.
But how should I handle that for the Centos image?
During the installation using Packer, there`s a reboot required...
Related
If you are developing Python web services for local network (servers is totally offline from the web) and the only way to add files to the server is through Flash drivers so using pip for Python packages or npm for node packages is such a headache and gets in a lot of dependencies issues and build issues .. so what is the proper way of dealing with such environment so development and deployment would be easier?
there are 2 approaches which you can take:
download all your dependencies locally and ship them to the remote server. this includes all the pip and npm packages. pay attention to the python\nodejs\operating system versions and architecture.
use docker to create an image, which packs everything. then ship the image to the remote server and finally spin-up a container based on that image.
You can use Pypicache to run your own pip servers and let it to cache your dependencies wherever you have an internet connection (where you are developing the application).
Then you can copy the whole pypicache folder on your flash drive and run the server wherever you want and use the cached packages inside it. the good point is in some environments that you can get a network connection for a limited time, having a pypicache is useful because it can download whatever all of the dependencies that your python applications need, and each instance would download and install the dependencies from the offline pip server by providing a simple switch in the command line. Here is an example:
pip install -i http://localhost:8080/simple somepackage
More Information - pypicache
I have to build and push docker image via gitlab-ci. I have gone through the official document.
https://docs.gitlab.com/ee/ci/docker/using_docker_build.html
I want to adopt shell method but my issue is I already had a working gitrunner on my server machine. So what the procedure for it. If I tried to re-register the git runner on the same machine. will it impact the old one?
Thanks in advance.
Assuming that you installed gitlab-runner as a system service and not inside a container, you can easily register another shell runner on your server using the command gitlab-ci-multi-runner register.
This is indirectly confirmed by the advanced configuration documentation, which states that the config.toml of the gitlab-runner service may contain multiple [[runner]] sections.
Note: To allow the shell runner to build docker images, you will need to add the gitlab-runner user to the docker group, e.g.:
sudo gpasswd --add gitlab-runner docker
When using packer to create a virtual box centos 7 VM does packer take care of installing the guest additions?
I have looked at several examples and one of them does the guest additions mounting and installing in the packer json file, is this necessary?
When I run the build I can see that packer takes care of downloading the guest additions but I am unsure if it installs them.
It doesn't install them. That you need to take care of yourself. It will download them to the host and upload it to the guest or attach it as CD.
I am running self-hosted GitLab CE with a GitLab-runner and a Docker executor. I want to build a binary for AWS Lambda, so I'm using the amazonlinux:latest image for my Docker executor.
Of course, not all packages that I need for building are available in the base amazonlinux image, so I install them via yum. Unfortunately, cmake is not available for Amazon Linux, so I build it from source.
At the moment, this takes place every time the pipeline runs, which is not optimal because cmake takes a relatively long time to build (compared to the binary I actually want to build).
My general question is: is there a clean and reproducible way to prepare an image for building, which is then used as base image for GitLab CI? Since I'm relatively new to Docker and friends, is the correct way to go to create an image locally on the runner host and use that in my gitlab-ci.yml? Or should I put it in a registry (probably even GitLab's own container registry?)
Yes there is.
Nothing is stopping you from creating an image through a Dockerfile that does all the yum installs and then pushes the image you build to a (private) Docker registry. Look at it as 'extending' the Amazon image and save it for future usage.
Since I don't expect it to be to exiting (it will not yet contain any application code) you can also store it for free on Docker Hub.
Custom image
So an example Dockerfile:
FROM amazonlinux:latest
RUN yum install <packages>
RUN <commands for cmake>
Then you build your custom amazonlinux-custom image with:
docker build -t mydockerhubuser/amazonlinux-custom:latest .
And push it to Docker Hub (after docker login):
docker push mydockerhubuser/amazonlinux-custom:latest
Gitlab CI usage
In your .gitlab-ci.yml you replace the image: amazonlinux:latest part that defines your job image with image: mydockerhubuser/amazonlinux-custom:latest so you don't have to install all your deps.
Note
Amazon will often rebuild its amazonlinux:latest image and push it to Docker Hub. Using a custom image based on theirs you will have to take in account the following:
You will need to rebuild your image often too, to stay up to date with patches etc.
It may be smarter to use e.g. a fixed version like FROM: amazonlinux:2017.09 to avoid major version changes you don't expect.
Im trying to setup Jenkins to run tests on a Virtual Machine but im not to sure how to proceed.
What id like to be able to do is to get Jenkins to build the environment on the vm and then have Jenkins execute the test scripts on the vm environment. After the tests have passed/failed id then like Jenkins to clean the database and pull down the virtual environment.
Server box - Windows 7
Virtaul Machine - VMWare
So what im looking for is some information or tutorials on how to implement the above. It would also be helpful if you could recommend what Jenkins plugins I can use to implement the above or if you want to go above and beyond can you outline the steps needed to achieve the above.
Any help would be appreciated.
I'm doing just that in my environment using the vSphere Cloud Plugin. Here's a basic step-by-step guide:
Install the plugin
Configure your ESX/ESXi server as a new "vSphere Cloud"
Create a new Jenkins node, of type "Slave virtual computer running under vSphere Cloud" (which becomes available after Installing the plugin).
When configuring the new node, optionally specify a snapshot name. This will revert the VM to this snapshot when the node launches.
Use the node in a pipeline script: node("node-name-or-label") { ...your code here... }
I use the method above with about 10 Windows nodes, reverting each to a "Clean" snapshot to start each build with a known state.