public/pipglr
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

doc: Format README.md

Browse Source
This commit is contained in:
Gabriel Nützi
2024-04-02 18:40:01 +02:00
parent 94e1055c5e
commit 2cd27dfae9
1 changed files with 132 additions and 130 deletions
Download Patch File Download Diff File Expand all files Collapse all files

262
README.md
View File

@@ -1,79 +1,84 @@
# Podmand-In-Podman Gitlab Runner
This project provides a Gitlab Runner which runs inside a container launched
with `podman`. The Gitlab Runner itself uses an independent `podman` instance
inside to launch jobs.
## Overview ## Overview
This container image is built daily from this `Containerfile`, and This container image is built daily from this `Containerfile`, and made
made available as: available as:
* `registry.gitlab.com/qontainers/pipglr:latest` - `registry.gitlab.com/qontainers/pipglr:latest`
-or- -or-
* `registry.gitlab.com/qontainers/pipglr:<version>` - `registry.gitlab.com/qontainers/pipglr:<version>`
It's purpose is to provide an easy method to execute a GitLab runner, It's purpose is to provide an easy method to execute a GitLab runner, to service
to service CI/CD jobs for groups and/or repositories on CI/CD jobs for groups and/or repositories on [gitlab.com](https://gitlab.com).
[gitlab.com](https://gitlab.com). It comes pre-configured to utilize It comes pre-configured to utilize the gitlab-runner app to execute within a
the gitlab-runner app to execute within a rootless podman container, rootless podman container, nested inside a rootless podman container.
nested inside a rootless podman container.
This is intended to provide additional layers of security for the host, This is intended to provide additional layers of security for the host, when
when running potentially arbitrary CI/CD code. Though, the ultimate running potentially arbitrary CI/CD code. Though, the ultimate responsibility
responsibility still rests with the end-user to review the setup and still rests with the end-user to review the setup and configuration relative to
configuration relative to their own security situation/environment. their own security situation/environment.
**Note**: While this can run entirely under a regular user, it will require **Note**: While this can run entirely under a regular user, it will require root
root access for the first two setup steps (below). access for the first two setup steps (below).
### Operation ### Operation
This image leverages the podman `runlabel` feature heavily. Several This image leverages the podman `runlabel` feature heavily. Several labels are
labels are set on the image to support easy registration and execution set on the image to support easy registration and execution of the runner
of the runner container. While it's possible to use the container container. While it's possible to use the container with your own command-line,
with your own command-line, it's highly recommended to base them it's highly recommended to base them off of one of the labels. See the examples
off of one of the labels. See the examples below for more information. below for more information.
***Note:*** Some older versions of podman don't support the **_Note:_** Some older versions of podman don't support the `container runlabel`
`container runlabel` sub-command. If this is the case, you may simulate sub-command. If this is the case, you may simulate it with the following,
it with the following, substituting `<label>` with one of the predefined substituting `<label>` with one of the predefined values (i.e. `register`,
values (i.e. `register`, `setupconfig`, etc.): `setupconfig`, etc.):
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ eval $(podman inspect --format=json $IMAGE | jq -r .[].Labels.<label>) $ eval $(podman inspect --format=json $IMAGE | jq -r .[].Labels.<label>)
``` ```
#### Persistent containers (step 1) #### Persistent Containers (step 1)
By default on many distributions, regular users aren't permitted to leave By default on many distributions, regular users aren't permitted to leave
background processes running after they log out. Since this is likely background processes running after they log out. Since this is likely desired
desired for running the pipglr container long-term, `systemd` needs to be for running the pipglr container long-term, `systemd` needs to be configured to
configured to override this policy. For this, you (`$USER`) will need override this policy. For this, you (`$USER`) will need root access on the
root access on the system. system.
```bash ```bash
$ sudo loginctl enable-linger $USER $ sudo loginctl enable-linger $USER
``` ```
Side-effect: This will allow your user to persist other user-level systemd Side-effect: This will allow your user to persist other user-level systemd
services as well. For example `podman.socket` is handy to enable for services as well. For example `podman.socket` is handy to enable for
`podman remote` access. You could also [setup `podman remote` access. You could also
quadlet](https://www.redhat.com/sysadmin/quadlet-podman) or a systemd unit [setup quadlet](https://www.redhat.com/sysadmin/quadlet-podman) or a systemd
so pipglr starts up on system boot. unit so pipglr starts up on system boot.
#### Expanded user-namespace (step 2) ***This is probably important*** #### Expanded User-Namespace (step 2) **_This is probably important_**
As an added protection/safety measure, pipglr excludes three UID/GIDs As an added protection/safety measure, pipglr excludes three UID/GIDs from being
from being used by job-level containers. One for `root`, another for used by job-level containers. One for `root`, another for `runner` and a third
`runner` and a third for `podman`. However, some container images for `podman`. However, some container images you may want to use for jobs
you may want to use for jobs (mainly Debian/Ubuntu), assign one/more (mainly Debian/Ubuntu), assign one/more essential users a high UID/GID value
essential users a high UID/GID value (like `65535`). (like `65535`).
At the same time, most distributions also set `65536` as the default maximum At the same time, most distributions also set `65536` as the default maximum
number (including ID `0`) of IDs to allocate for user-namespaces (via `/etc/login.defs`). This number (including ID `0`) of IDs to allocate for user-namespaces (via
creates a problem you won't realize until the runner actually picks up a job `/etc/login.defs`). This creates a problem you won't realize until the runner
😞 The main symptom of this issue will be messages in the pipglr containers log, actually picks up a job 😞 The main symptom of this issue will be messages in
similar to (abbreviated): the pipglr containers log, similar to (abbreviated):
``` ```text
...cut... ...cut...
running `/usr/bin/newuidmap ...cut...`: newuidmap: write to uid_map failed: Operation not permitted running `/usr/bin/newuidmap ...cut...`: newuidmap: write to uid_map failed: Operation not permitted
Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1 Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1
@@ -82,71 +87,71 @@ Error: cannot set up namespace using "/usr/bin/newuidmap": exit status 1
or or
``` ```text
E: setgroups 65534 failed - setgroups (22: Invalid argument) E: setgroups 65534 failed - setgroups (22: Invalid argument)
``` ```
***The good news is, working around this is relatively simple:*** **_The good news is, working around this is relatively simple:_**
As root, edit the two files `/etc/subuid` and `/etc/subgid` to expand the As root, edit the two files `/etc/subuid` and `/etc/subgid` to expand the by 3
by 3 IDs. For example assuming a user running the pipglr container is IDs. For example assuming a user running the pipglr container is called
called `johndoe`, the contents of these files should be edited to allocate `johndoe`, the contents of these files should be edited to allocate `65539` IDs
`65539` IDs like: like:
`jogndoe:<some number>:65539` `johndoe:<some number>:65539`
Where `<some number>` was set by your OS when the `johndoe` user was created Where `<some number>` was set by your OS when the `johndoe` user was created
(you can ignore this). Only the last number needs to be increased. This (you can ignore this). Only the last number needs to be increased. This change
change will be effective on next login, or immediately by running: will be effective on next login, or immediately by running:
`podman system migrate` `podman system migrate`
*Note:* This will stop any currently running containers. _Note:_ This will stop any currently running containers.
#### Runner registration (step 3) #### Runner Registration (step 3)
All runners must be connected to a project or group runner configuration All runners must be connected to a project or group runner configuration on your
on your gitlab instance (or `gitlab.com`). This is done using a special gitlab instance (or `gitlab.com`). This is done using a special registration
registration *runlabel*. The command can (and probably should) be run _runlabel_. The command can (and probably should) be run more than once (using
more than once (using the same `config.toml`) to configure and register the same `config.toml`) to configure and register multiple runners. This is
multiple runners. This is necessary for the *pipglr* container to execute necessary for the _pipglr_ container to execute multiple jobs in parallel. For
multiple jobs in parallel. For example, if you want to support running example, if you want to support running four jobs at the same time, you would
four jobs at the same time, you would use the `register` *runlabel* use the `register` _runlabel_ four times.
four times.
Before using the `register` *runlabel*, you must set your unique Before using the `register` _runlabel_, you must set your unique _registration_
*registration* (a.k.a. *activation*) token as a podman *secret*. This (a.k.a. _activation_) token as a podman _secret_. This secret may be removed
secret may be removed once the registration step is complete. The once the registration step is complete. The **`<actual registration token>`**
**<actual registration token>** value (below) should be replaced with value (below) should be replaced with the value obtained from the `runners`
the value obtained from the "runners" settings page of a gitlab settings page of a gitlab group or project's _CI/CD Settings_. Gitlab version 16
group or project's *CI/CD Settings*. Gitlab version 16 and later and later refers to this value as an _activation_ token, but the usage is the
refers to this value as an *activation* token, but the usage is the same. same.
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ echo '<actual registration token>' | podman secret create REGISTRATION_TOKEN - $ echo '<actual registration token>' | podman secret create REGISTRATION_TOKEN -
``` ```
Next, ***a blank `config.toml` file*** needs to be created. Without this, the Next, **_a blank `config.toml` file_** needs to be created. Without this, the
`reigster` *runlabel* will return a permission-denied error. Once the empty `reigster` _runlabel_ will return a permission-denied error. Once the empty
`config.toml` file is created, you may register one or more runners by repeating `config.toml` file is created, you may register one or more runners by repeating
the registration *runlabel* as follows: the registration _runlabel_ as follows:
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ touch ./config.toml # important: file must exist, even if empty. $ touch ./config.toml # important: file must exist, even if empty.
$ podman container runlabel register $IMAGE $ podman container runlabel register $IMAGE
...repeat as desired... # ...repeat as desired...
$ podman secret rm REGISTRATION_TOKEN # if desired $ podman secret rm REGISTRATION_TOKEN # if desired
``` ```
#### Runner Configuration (step 4) #### Runner Configuration (step 4)
During the registration process (above), a boiler-plate (default) `config.toml` file During the registration process (above), a boiler-plate (default) `config.toml`
will be created/updated for you. At this point you may edit the configuration file will be created/updated for you. At this point you may edit the
if desired before committing it as a *podman secret*. Please refer to the configuration if desired before committing it as a _podman secret_. Please refer
[gitlab runner documentation](https://docs.gitlab.com/runner/configuration/) to the
for details. [gitlab runner documentation](https://docs.gitlab.com/runner/configuration/) for
details.
```bash ```bash
$ $EDITOR ./config.toml # if desired $ $EDITOR ./config.toml # if desired
@@ -155,15 +160,15 @@ $ rm ./config.toml # if desired
``` ```
This may be necessary, for example, to increase the default `concurrency` value This may be necessary, for example, to increase the default `concurrency` value
to reflect the number of registered runners. If you need to edit this file to reflect the number of registered runners. If you need to edit this file after
after committing it as a secret, there's committing it as a secret, there's
[ a `dumpconfig` *runlabel* for that](README.md#configuration-editing). [a `dumpconfig` _runlabel_ for that](README.md#configuration-editing).
#### Volume setup (step 5) #### Volume Setup (step 5)
Since several users are utilized inside the container volumes must be Since several users are utilized inside the container volumes must be
specifically configured to permit access. This is done using several specifically configured to permit access. This is done using several _runlabels_
*runlabels* as follows: as follows:
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
@@ -171,16 +176,17 @@ $ podman container runlabel setupstorage $IMAGE
$ podman container runlabel setupcache $IMAGE $ podman container runlabel setupcache $IMAGE
``` ```
Note: These volumes generally do not contain any critical operational data, Note: These volumes generally do not contain any critical operational data, they
they may be re-created anytime to quickly free up host disk-space if may be re-created anytime to quickly free up host disk-space if it's running
it's running low. Simply remove them with the command low. Simply remove them with the command
`podman volume rm pipglr-storage pipglr-cache`. Then reuse the `setupstorage` `podman volume rm pipglr-storage pipglr-cache`. Then reuse the `setupstorage`
and `setupcache` *runlabels* as in the above example. and `setupcache` _runlabels_ as in the above example.
#### Runner Startup (step 6) #### Runner Startup (step 6)
With the runner configuration saved as a Podman secret, and the runner volumes With the runner configuration saved as a Podman secret, and the runner volumes
created, the GitLab runner container may be launched with the following commands: created, the GitLab runner container may be launched with the following
commands:
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
@@ -189,11 +195,10 @@ $ podman container runlabel run $IMAGE
### Configuration Editing ### Configuration Editing
The gitlab-runner configuration contains some sensitive values which The gitlab-runner configuration contains some sensitive values which should be
should be protected. The pipglr container assumes the entire configuration protected. The pipglr container assumes the entire configuration will be passed
will be passed in as a Podman secret. This makes editing it slightly in as a Podman secret. This makes editing it slightly convoluted, so a handy
convoluted, so a handy *runlabel* `dumpconfig` is available. _runlabel_ `dumpconfig` is available. It's intended use is as follows:
It's intended use is as follows:
```bash ```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest" $ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
@@ -206,17 +211,17 @@ $ rm ./config.toml # if desired
### Debugging ### Debugging
The first thing to check is the container output. This shows three things: The first thing to check is the container output. This shows three things:
Systemd, Podman, and GitLab-Runner output. For example: Systemd, Podman, and GitLab-Runner output. For example:
```bash ```bash
$ podman logs --since 0 pipglr $ podman logs --since 0 pipglr
``` ```
Next, try running a pipglr image built with more verbose logging. Both Next, try running a pipglr image built with more verbose logging. Both the
the `runner.service` and `podman.service` files have a `log-level` option. `runner.service` and `podman.service` files have a `log-level` option. Simply
Simply increase one or both to the "info", or "debug" level. Start the increase one or both to the `info`, or `debug` level. Start the debug container,
debug container, and reproduce the problem. and reproduce the problem.
## Building ## Building
@@ -226,39 +231,36 @@ This image may be built simply with:
$ podman build -t registry.gitlab.com/qontainers/pipglr:latest . $ podman build -t registry.gitlab.com/qontainers/pipglr:latest .
``` ```
This will utilize the latest stable version of podman and the latest This will utilize the latest stable version of podman and the latest stable
stable version of the gitlab runner. version of the gitlab runner.
### Build-args ### Build-Arguments
Several build arguments are available to control the output image: Several build arguments are available to control the output image:
* `PRUNE_INTERVAL` - A systemd.timer compatible `OnCalendar` value that - `PRUNE_INTERVAL`: A systemd.timer compatible `OnCalendar` value that
determines how often to prune Podman's storage of disused containers and determines how often to prune Podman's storage of disused containers and
images. Defaults to "daily", but should be adjusted based on desired images. Defaults to `daily`, but should be adjusted based on desired
caching-effect balanced against available storage space and job caching-effect balanced against available storage space and job execution
execution rate. rate.
* `RUNNER_VERSION` - Allows specifying an exact gitlab runner version. - `RUNNER_VERSION`: Allows specifying an exact gitlab runner version. By default
By default the `latest` is used, assuming the user is building a tagged the `latest` is used, assuming the user is building a tagged image anyway.
image anyway. Valid versions may be found on the [runner Valid versions may be found on the
release page](https://gitlab.com/gitlab-org/gitlab-runner/-/releases). [runner release page](https://gitlab.com/gitlab-org/gitlab-runner/-/releases).
* `TARGETARCH` - Supports inclusion of non-x86_64 gitlab runners. This - `TARGETARCH`: Supports inclusion of non-x86_64 gitlab runners. This value is
value is assumed to match the image's architecture. If using the assumed to match the image's architecture. If using the `--platform` build
`--platform` build argument, it will be set automatically. Note: argument, it will be set automatically. Note: as of this writing, only `amd64`
as of this writing, only `amd64` and `arm64` builds of the gitlab-runner and `arm64` builds of the gitlab-runner are available.
are available. - `GITLAB_URL`: Defaults to `https://gitlab.com/` but can be set to point to a
* `GITLAB_URL` - Defaults to 'https://gitlab.com/' but can be set to point self hosted instance of Gitlab.
to a self hosted instance of Gitlab. - `NESTED_PRIVILEGED`: Defaults to `true`, may be set `false` to prevent nested
* `NESTED_PRIVILEGED` - Defaults to 'true', may be set 'false' to prevent containers running in `--privileged` mode. This will affect the ability to
nested containers running in `--privileged` mode. This will affect build container images in CI jobs using tools like podman or buildah.
the ability to build container images in CI jobs using tools like
podman or buildah.
### Environment variables ### Environment Variables
Nearly every option to every gitlab-runner sub-command may be specified via Nearly every option to every gitlab-runner sub-command may be specified via
environment variable. Some of these are set in the `Containerfile` for environment variable. Some of these are set in the `Containerfile` for the
the `register` *runlabel*. If you need to set additional runtime `register` _runlabel_. If you need to set additional runtime env. vars., please
env. vars., please do so via additional `Environment` optionns in the do so via additional `Environment` optionns in the `runner.service` file. See
`runner.service` file. See the *systemd.nspawn* man page for important the _systemd.nspawn_ man page for important value-format details.
value-format details.