For normal operations, the `runner` user's `.gitlab-runner` directory
has a symlink for (read-only) `config.toml` ->
`/var/run/secrets/config.toml`. However, for during registration
attempts to write to this volume-mounted file result in a
`PANIC: write: permission denied` error. Fix this by running the
registration as (namespaced) `root` user, and using the default config.
file location of `/etc/gitlab-runner`.
Signed-off-by: Chris Evich <cevich@redhat.com>
The README did not make it clear enough that a blank `config.toml` file
needs to be pre-created before runner registration. Otherwise the
`register` *runlabel* will silently fail to bind-mount the file into the
proper location inside the container.
While we're at it, also add a small example of why it may be a good idea
to edit the `config.toml` file before commiting it as a secret.
Signed-off-by: Chris Evich <cevich@redhat.com>
When running in the background without a full-blown init system,
`podman system service` will leak `conmon` processes for every
gitlab-runner job that executes via the docker socket API. These
`conmon` processes almost immediately becomes zombies, and are never
cleaned up. Eventually the zombies will consume all available PIDs.
Many attempts to fix this in various ways have all failed. In all cases
the GitLab Runner process will start behaving strangely (or fail
completely) after an amount of time dependent on its usage executing
jobs.
Fix this by entirely reimplementing *pipglr* to utilize systemd and a
pair of lingering user-slices. One for podman, another for the gitlab
runner. Include a systemd timer service to affect runner cleanup,
periodically. Also update documentation and examples accordingly.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
Within a container, caps need to be set on /usr/bin/new{uid,gid}map
which reflect the (inner) podman user's root namespaced ID of 10000.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
Recently, I believe an update or packaging problem has been causing
podman to throw errors like:
```
level=error msg="running `/usr/bin/newuidmap ...`: newuidmap: open of
uid_map failed: Permission denied\n"
```
This seems to have something to do with the shadow-utils package, which
owns this binary. I've examined the file attribuites and permissions
along with /etc/sub{uid,gid} contents. The only thing that seems to
resolve the issue is reinstalling shadow-utils. Attempt that fix here
and hope it clears up the problem (present in v2.0.0)
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
* Add note about volume-mounts being cumulative with base-image
* Fix register & run labels to use (correct) base image's
`/home/podman/.local/share/containers/` instead of defining
a new (wrong/useless) `storage` volume.
* Fix register & run labels to mask over `/var/lib/containers`
with a read-only tmpfs to block any nested rootful use of
podman as a security precaution.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
This reverts commit 4c1af406f7. For
whatever reason, trying to build this using the community runners always
results in an out-of-diskspace error.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
This vastly improves build performance at the cost of some extra
registry server storage (which there is plenty of). Note: The cache
TTL needs to be coordinated with the registry server's tag-cleanup
policy.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
The podman base-image is intended to support running nested-podman both
root and rootless. Since pipglr only ever runs rootless, eliminate the
nested usernamespace mapping needed to support nested-root usage.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
They were preventing builds on main and on tags. Rather than try and
debug them, just kill them as they're mostly unnecessary for this repo.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
Observations show the runtime is about the same (since performance is mostly
storage-io bound. However, buildah images are more compact and there
are simpler possibilities for build-caching available for future use.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
The function was defined but never called, resulting in immediate exit
of the maintenance script. Fix this, also add a configuration build-arg and
ENV to control the cleaning interval.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
This CI environment is using a busybox `sh` so doesn't support all the
advanced features of bash. Reimplement IMAGE_TAG processing so it
functions as intended.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
It's often hard to debug/maintain/improve a job when you can't observe
any of the (many!) auto-generated CI env. vars. Print them all out on
every job.
Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
