6cb20272e4
When running in the background without a full-blown init system, `podman system service` will leak `conmon` processes for every gitlab-runner job that executes via the docker socket API. These `conmon` processes almost immediately becomes zombies, and are never cleaned up. Eventually the zombies will consume all available PIDs. Many attempts to fix this in various ways have all failed. In all cases the GitLab Runner process will start behaving strangely (or fail completely) after an amount of time dependent on its usage executing jobs. Fix this by entirely reimplementing *pipglr* to utilize systemd and a pair of lingering user-slices. One for podman, another for the gitlab runner. Include a systemd timer service to affect runner cleanup, periodically. Also update documentation and examples accordingly. Signed-off-by: Chris Evich <chris_gitlab@icuc.me>
82 lines
3.8 KiB
Docker
82 lines
3.8 KiB
Docker
FROM quay.io/centos/centos:stream9
|
|
|
|
ADD /setup.sh /xpackages.txt /root/
|
|
ADD /containers.conf /home/podman/.config/containers/containers.conf
|
|
ADD /podman.service /podman.socket /prune.service /prune.timer /home/podman/.config/systemd/user/
|
|
ADD /runner.service /home/runner/.config/systemd/user/
|
|
ADD kmsglog.conf /etc/systemd/system.conf.d/
|
|
|
|
# Allow image-builders to choose another version becides "latest" should
|
|
# an incompatible change be introduced.
|
|
ARG RUNNER_VERSION=latest
|
|
|
|
# Permit building containers for alternate architectures. At the time
|
|
# of this commit, only 'arm64' is available.
|
|
ARG TARGETARCH=amd64
|
|
|
|
# Allow image-builders to choose an alternate nested-container pruning cycle.
|
|
# For most people the default is probably fine. This setting is dependent
|
|
# on the number and frequency of jobs run, along with the amount of disk-space
|
|
# available for both /cache and /home/podman/.local/share/containers volumes.
|
|
ARG PRUNE_INTERVAL=daily # see systemd.timer for allowable values
|
|
|
|
# All-in-one packaging/image-setup script to keep things simple.
|
|
RUN PRUNE_INTERVAL=${PRUNE_INTERVAL} \
|
|
RUNNER_VERSION=${RUNNER_VERSION} \
|
|
bash /root/setup.sh
|
|
|
|
VOLUME /cache /home/podman/.local/share/containers
|
|
ENTRYPOINT /lib/systemd/systemd
|
|
|
|
# Gitlab-runner configuration options, may be freely overridden at
|
|
# container image build time.
|
|
ARG DEFAULT_JOB_IMAGE=registry.fedoraproject.org/fedora-minimal:latest
|
|
# Run nested containers in --privileged mode - required to allow building
|
|
# container images using podman or buildah. Otherwise may be set 'false'.
|
|
ARG NESTED_PRIVILEGED=true
|
|
|
|
# The registration runlabel may be called multiple times to register more than
|
|
# one runner. Each expects a REGISTRATION_TOKEN secret to be pre-defined and
|
|
# the file './config.toml' to exist (may be empty). A local-cache volume
|
|
# '/cache' is configured for bind-mounting into all interrior-containers
|
|
# for container-runtime use, as recommended by the docs. Other settings
|
|
# may be changed if you know what you're doing.
|
|
LABEL register="podman run -it --rm \
|
|
--secret=REGISTRATION_TOKEN,type=env \
|
|
-v ./config.toml:/home/runner/.gitlab-runner//config.toml:Z \
|
|
-e REGISTER_NON_INTERACTIVE=true \
|
|
-e CI_SERVER_URL=https://gitlab.com/ \
|
|
-e RUNNER_NAME=pipglr \
|
|
-e RUNNER_EXECUTOR=docker \
|
|
-e RUNNER_SHELL=bash \
|
|
-e REGISTER_MAINTENANCE_NOTE=Podman-In-Podman-GitLab-Runner \
|
|
-e DOCKER_HOST=unix:///home/runner/podman.sock \
|
|
-e DOCKER_IMAGE=${DEFAULT_JOB_IMAGE} \
|
|
-e DOCKER_CACHE_DIR=/cache \
|
|
-e DOCKER_VOLUMES=/cache \
|
|
-e DOCKER_NETWORK_MODE=host \
|
|
-e DOCKER_PRIVILEGED=${NESTED_PRIVILEGED} \
|
|
--user runner \
|
|
--entrypoint=/usr/bin/gitlab-runner \$IMAGE register"
|
|
# Additionally, the nested-podman storage volumes must be pre-created with
|
|
# 'podman' UID/GID values to allow nested containers access.
|
|
LABEL setupstorage="podman volume create --opt o=uid=1000,gid=1000 pipglr-storage"
|
|
# Lastly, the gitlab-runner will manage container-cache in this directory,
|
|
# which will also be bind-mounted into every container. So it must be
|
|
# writable by both 'podman' user and 'runner' group.
|
|
LABEL setupcache="podman volume create --opt o=uid=1000,gid=1001 pipglr-cache"
|
|
# Helper to extract the current configuration secret to allow editing.
|
|
LABEL dumpconfig="podman run -it --rm \
|
|
--secret config.toml --entrypoint=/bin/cat \
|
|
\$IMAGE /var/run/secrets/config.toml"
|
|
# Executing the runner container depends on the config.toml secret being
|
|
# set (see above) and two volumes existing with correct permissions set.
|
|
# Note: The contents of the volumes are not critical, they may be removed
|
|
# and re-created (see above) to quickly free-up disk space.
|
|
LABEL run="podman run -dt --name pipglr \
|
|
--secret config.toml,uid=1001,gid=1001 \
|
|
-v pipglr-storage:/home/podman/.local/share/containers \
|
|
-v pipglr-cache:/cache \
|
|
--systemd true --privileged \
|
|
--device /dev/fuse \$IMAGE"
|