pipglr/README.md

## Overview

This container image is built daily from this `Containerfile`, and
made available as:

* `registry.gitlab.com/qontainers/pipglr:latest`

-or-

* `registry.gitlab.com/qontainers/pipglr:<version>`

It's purpose is to provide an easy method to execute a GitLab runner,
to service CI/CD jobs for groups and/or repositories on
[gitlab.com](https://gitlab.com).  It comes pre-configured to utilize
the gitlab-runner app to execute within a rootless podman container,
nested inside a rootless podman container.

This is intended to provide additional layers of security for the host,
when running potentially arbitrary CI/CD code.  Though, the ultimate
responsibility still rests with the end-user to review the setup and
configuration relative to their own security situation/environment.

### Operation

This image leverages the podman `runlabel` feature heavily.  Several
labels are set on the image to support easy registration and execution
of the runner container.  While it's possible to use the container
with your own command-line,  it's highly recommended to base them
off of one of the labels.  See the examples below for more information.

***Note:*** Some older versions of podman don't support the
`container runlabel` sub-command.  If this is the case, you may simulate
it with the following, substituting `<label>` with one of the predefined
values (i.e. `register`, `setupconfig`, etc.):

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ eval $(podman inspect --format=json $IMAGE | jq -r .[].Labels.<label>)
```

#### Runner registration (step 1)

All runners must be connected to a project or group runner configuration
on your gitlab instance (or `gitlab.com`).  This is done using a special
registration *runlabel*.  The command can (and probably should) be run
more than once (using the same `config.toml`) to configure and register
multiple runners.  This is necessary for the *pipglr* container to execute
multiple jobs in parallel.  For example, if you want to support running
four jobs at the same time, you would use the `register` *runlabel*
four times.

Before using the `register`  *runlabel*, you must set your unique
*registration* (a.k.a. *activation*) token as a podman *secret*.  This
secret may be removed once the registration step is complete.  The
**<actual registration token>** value (below) should be replaced with
the value obtained from the "runners" settings page of a gitlab
group or project's *CI/CD Settings*.  Gitlab version 16 and later
refers to this value as an *activation* token, but the usage is the same.

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ echo '<actual registration token>' | podman secret create REGISTRATION_TOKEN -
```

Next, ***a blank `config.toml` file*** needs to be created.  Without this, the
`reigster` *runlabel* will return a permission-denied error.  Once the empty
`config.toml` file is created, you may register one or more runners by repeating
the registration *runlabel* as follows:

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ touch ./config.toml  # important: file must exist, even if empty.
$ podman container runlabel register $IMAGE
...repeat as desired...
$ podman secret rm REGISTRATION_TOKEN  # if desired
```

#### Runner Configuration (step 2)

During the registration process (above), a boiler-plate (default) `config.toml` file
will be created/updated for you.  At this point you may edit the configuration
if desired before committing it as a *podman secret*.  Please refer to the
[gitlab runner documentation](https://docs.gitlab.com/runner/configuration/)
for details.

```bash
$ $EDITOR ./config.toml  # if desired
$ podman secret create config.toml ./config.toml
$ rm ./config.toml  # if desired
```

This may be necessary, for example, to increase the default `concurrency` value
to reflect the number of registered runners.  If you need to edit this file
after commiting it as a secret, there's
[ a `dumpconfig`  *runlabel* for that](README.md#configuration-editing).

#### Volume setup (step 3)

Since several users are utilized inside the container volumes must be
specifically configured to permit access.  This is done using several
*runlabels* as follows:

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ podman container runlabel setupstorage $IMAGE
$ podman container runlabel setupcache $IMAGE
```

Note: These volumes generally do not contain any critical operational data,
they may be re-created anytime to quickly free up host disk-space if
it's running low.  Simply remove them with the command
 `podman volume rm pipglr-storage pipglr-cache`.  The reuse the `setupstorage`
and `setupcache` *runlabels* as in the above example.

#### Runner Startup (step 4)

With the runner configuration saved as a Podman secret, and the runner volumes
created, the GitLab runner container may be launched with the following commands:

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ podman container runlabel run $IMAGE
```

### Configuration Editing

The gitlab-runner configuration contains some sensitive values which
should be protected.  The pipglr container assumes the entire configuration
will be passed in as a Podman secret.  This makes editing it slightly
convoluted, so a handy *runlabel* `dumpconfig` is available.
It's intended use is as follows:

```bash
$ IMAGE="registry.gitlab.com/qontainers/pipglr:latest"
$ podman container runlabel dumpconfig $IMAGE > ./config.toml
$ $EDITOR ./config.toml
$ podman secret rm config.toml
$ podman secret create config.toml ./config.toml
$ rm ./config.toml  # if desired
```

### Debugging

The first thing to check is the container output.  This shows three things:
Systemd, Podman, and GitLab-Runner output.  For example:

```bash
$ podman logs --since 0 pipglr
```

Next, try running a pipglr image built with more verbose logging.  Both
the `runner.service` and `podman.service` files have a `log-level` option.
Simply increase one or both to the "info", or "debug" level.  Start the
debug container, and reproduce the problem.

## Building

This image may be built simply with:

```bash
$ podman build -t registry.gitlab.com/qontainers/pipglr:latest .
```

This will utilize the latest stable version of podman and the latest
stable version of the gitlab runner.

### Build-args

Several build arguments are available to control the output image:

* `PRUNE_INTERVAL` - A systemd.timer compatible `OnCalendar` value that
  determines how often to prune Podman's storage of disused containers and
  images.  Defaults to "daily", but should be adjusted based on desired
  caching-effect balanced against available storage space and job
  execution rate.
* `RUNNER_VERSION` - Allows specifying an exact gitlab runner version.
  By default the `latest` is used, assuming the user is building a tagged
  image anyway.  Valid versions may be found on the [runner
  release page](https://gitlab.com/gitlab-org/gitlab-runner/-/releases).
* `TARGETARCH` - Supports inclusion of non-x86_64 gitlab runners.  This
  value is assumed to match the image's architecture.  If using the
  `--platform` build argument, it will be set automatically.  Note:
  as of this writing, only `amd64` and `arm64` builds of the gitlab-runner
  are available.
* `NESTED_PRIVILEGED` - Defaults to 'true', may be set 'false' to prevent
  nested containers running in `--privileged` mode.  This will affect
  the ability to build container images in CI jobs using tools like
  podman or buildah.

### Environment variables

Nearly every option to every gitlab-runner sub-command may be specified via
environment variable.  Some of these are set in the `Containerfile` for
the `register` *runlabel*.  If you need to set additional runtime
env. vars., please do so via additional `Environment` optionns in the
`runner.service` file.  See the *systemd.nspawn* man page for important
value-format details.