Skip to content

STAC-25142 Move the agent CI to GitHub Actions and retire the GitLab pipelines - #444

Open
LouisParkin wants to merge 52 commits into
stackstate-7.78.2from
STAC-25142-agent-lint-unit
Open

STAC-25142 Move the agent CI to GitHub Actions and retire the GitLab pipelines#444
LouisParkin wants to merge 52 commits into
stackstate-7.78.2from
STAC-25142-agent-lint-unit

Conversation

@LouisParkin

@LouisParkin LouisParkin commented Jul 10, 2026

Copy link
Copy Markdown

What

Ports the stackstate-agent GitLab pipeline to GitHub Actions, for amd64 and arm64.

Workflow Covers
Lint and unit tests filename linting, inv check-mod-tidy, branded + unbranded unit suites
Binary builds agent and cluster-agent binaries, cluster-agent image build/publish/sign, multi-arch manifest
DEB package build omnibus DEB, branding verification, agent image build/publish/sign, GPG signing, pre-release apt repository, install script
Go dependency cache image reusable — bakes the module cache so the heavy jobs start warm instead of re-downloading
Cerberus notify reusable — reports release-branch failures to Slack

Each top-level workflow ends in a CI success (...) aggregate, and those three are the required checks on the base branch. A matrix job cannot be required directly by name, so the aggregate is what branch protection can pin.

Nightly

GitLab schedule 20682 builds the release branch every weekday evening (22:00 Europe/Amsterdam; GitHub cron is UTC-only, so the trigger uses 20:00 UTC and drifts an hour in winter). The build and DEB workflows carry an equivalent schedule: trigger so archiving GitLab does not end nightly builds silently. The unit suites are excluded — two ~2h jobs that already run on the PR and on the push.

Publication is widened to schedule alongside push, otherwise the nightly would build and discard everything, and a nightly failure would reach nobody. Image tags are the commit SHA, so an unchanged branch republishes an identical tag; the signal is drift in base images and dependencies, not fresh artifacts.

GitLab pipelines are removed

The four StackState-owned GitLab pipeline files are deleted, since these workflows replace them. Scheduled runs were the only pipelines GitLab had executed since 27 July, so the nightly above is what was keeping it alive.

Left in place deliberately: .gitlab/ (129 upstream files that ~15 upstream tasks/*.py modules read), .gitlab-scripts/setup_artifact_registry.sh (called by the DEB build here, tasks/omnibus.py and an omnibus software definition), and .ci-builders/. Docs are repointed at the workflows.

Publication is push-only

Every publishing job is gated on github.event_name == 'push'. A PR builds, smoke-tests and scans the images and the DEB, then throws them away. Jobs needing registry credentials skip on fork PRs.

The 256 go.mod / go.sum files

One commit, STAC-25466. inv check-mod-tidy is now a gate and the workspace manifests did not survive it. No version was picked by hand; this is what go mod tidy produces.

Validation

All three required aggregates green on 3e2a23fe, including the first full arm64 run. Branch is up to date with the base.

Jira: https://stackstate.atlassian.net/browse/STAC-25142
Sub-tasks merged into this branch: STAC-24889, 25429, 25457, 25459, 25461, 25466, 25494, 25498, 25500, 25536, 25575, 25583

@LouisParkin
LouisParkin changed the base branch from stackstate-7.71.2 to stackstate-7.78.2 July 10, 2026 15:20
@LouisParkin
LouisParkin force-pushed the STAC-25142-agent-lint-unit branch from 3cd1a54 to 7308812 Compare July 10, 2026 15:22
@LouisParkin
LouisParkin force-pushed the STAC-25142-agent-lint-unit branch from 76349d9 to c2e9da2 Compare July 27, 2026 09:25
LouisParkin and others added 8 commits July 28, 2026 11:39
…ration phase 1)

Ports filename_linting + unbranded/branded unit_tests from .gitlab-ci-agent.yml
to GitHub Actions, tracking the active stackstate-7.78.2 pipeline.

- filename-linting on `docker` (cheap gate; only needs `git ls-files`).
- unbranded/branded unit tests on `xlarge` inside the datadog_build image
  (tag 7af9194f), pulled via the read-only quay proxy.
- 7.78.2 deltas vs 7.71.2: `--major-version` dropped from agent.build/inv test
  (removed upstream in DD 7.78), and branded runs scripts/verify_branding_literals.sh
  after fix_branding.sh.

Test-execution policy (architect directive — minimize redundant runs): each
suite runs once. filename-lint on every PR; the two heavy ~2h suites on
pull_request, deduped via concurrency cancel-in-progress. They move to
merge_group (run once at land) at phase 6; master/version-branch pushes then
build+publish without re-running tests.

Security: minimal contents: read, SHA-pinned checkout, persist-credentials
false, no PATs, registry-proxy secret only via container.credentials, fork PRs
skipped. Zizmor clean; actionlint clean apart from self-hosted labels.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…ic/xlarge-public)

Public repos can't use the shared docker/xlarge runners; use the isolated agent-public pool (pulumi-infra #227 + argocd-apps #115).
…on phase 2)

Port build_binaries and build_cluster_agent from .gitlab-ci-agent.yml to
.github/workflows/build-binaries.yml: build the branded (StackState) production
agent (+ rtloader) and cluster-agent in the datadog_build container on public
xlarge runners, once per change (pull_request), deduped via concurrency.

Carry over the GitLab pipeline speedups: self-contained jobs with no shared deps
prerequisite (the Go module cache is untransportable between jobs and shared
vendoring breaks the go.mod<->vendor reconcile), version.txt generated in-job
from git, and a runner-local persistent-cache follow-up noted for the real
warm-cache win.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…e image

Speeds up the migrated stackstate-agent GitHub Actions build/test jobs by
replacing each job's cold `go clean -modcache` + `inv deps` (go mod download +
tidy) reconcile with a warm GOMODCACHE delivered as a prebuilt CI image, keyed
by a content hash of the module graph. Mirrors the StackGraph
ci-metadata / build-ci-image pattern.

- .github/scripts/agent-godeps-cache-metadata.sh: content-addressed image tag
  from every go.mod/go.sum + go.work + modules.yml plus the Dockerfile/script
  themselves, so any graph or mechanism change rotates the tag.
- .github/docker/godeps-cache/Dockerfile: FROM datadog_build, bakes external
  modules into GOMODCACHE via a per-module `go mod download` loop.
- .github/workflows/godeps-cache.yml: reusable workflow (tag lookup + build only
  when missing); pushes to quay.io/stackstate (private), pulled by consumers
  through the registry.tooling quay proxy with the existing REGISTRY_* creds.
- build-binaries.yml / lint-and-unit-tests.yml: add the godeps-cache job, repoint
  the 4 heavy jobs to the cache image, drop the per-job modcache reset + inv deps
  (keep `go work sync` / `go work vendor` against the warm cache).

Zizmor clean (narrow unpinned-images ignore on the content-addressed container
refs, which are effectively digest-pinned but computed per run).

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin
LouisParkin force-pushed the STAC-25142-agent-lint-unit branch from c2e9da2 to c07d0c8 Compare July 28, 2026 09:40
The Cerberus Lambda already knows which channel to post to, so passing one
from the workflow is redundant. `notify.go` resolves the channel as:

    Channel: util.GetOrDefault(req.Context, "channel", s.Channel)

and `util.GetOrDefault` (internal/util/context.go:27) treats an empty or
whitespace value as absent, falling back to the Lambda's own SLACK_CHANNEL
environment variable. Since this repo does not hold the secret, the previous
code was sending `channel: ""` on every call and hitting that fallback
anyway -- the field never did anything.

Dropping it halves what pulumi-infra has to provision for this repo
(StackVista/pulumi-infra#258 now asks for CERBERUS_LAMBDA_URL alone), which
matters because org-level secrets default to visibility=private and are
therefore invisible to this PUBLIC repo.

The `channel` key is removed from the payload entirely rather than sent
empty; GetOrDefault makes the two equivalent, and omitting it states the
intent. A comment records how to reintroduce it if agent failures should
ever go somewhere other than the shared CI channel.

Verified: actionlint clean (only the pre-existing self-hosted runner-label
warnings), zizmor clean, and the jq payload still emits valid JSON in the
shape Cerberus expects.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
STAC-25461 Drop SLACK_CI_REPORT_CHANNEL from the Cerberus notify path
@LouisParkin

Copy link
Copy Markdown
Author

Added commit: Cerberus bearer token (STAC-25536)

Heads-up for anyone who already reviewed — one commit added on top: ed1f889f.

StackVista/cerberus#4 (STAC-24889) adds authentication to the Cerberus Lambda. The endpoint was previously completely unauthenticated — anyone who knew the URL could lock/unlock a protected branch or post as the Cerberus Slack bot. Every non-Slack request must now carry Authorization: Bearer <ssm:/cerberus/api-token>, compared constant-time in internal/auth/auth.goverifyBearer.

That PR is still open, so cerberus-notify.yml as reviewed works today. The moment it deploys, the unauthenticated POST gets a 401, curl --fail reddens the notify job, and no Slack message is sent — the exact silent-failure condition the job exists to remove.

Sending the header now is forward-compatible (the deployed Lambda ignores unknown headers), so this is correct either side of that deploy and needs no coordination. Fixing it here rather than as a follow-up PR because this branch hasn't merged yet.

Changes

  • cerberus-notify.yml: CERBERUS_API_TOKEN added to workflow_call.secrets + step env; bearer header on the POST; guard extended to both secrets.
  • The three callers (build-binaries, build-deb, lint-and-unit-tests) pass the secret through.
  • Dropped curl --verbose — it echoes request headers, which now include the token. GitHub would mask it, but not emitting it beats relying on masking.

CERBERUS_API_TOKEN already exists as a repo-level secret alongside CERBERUS_LAMBDA_URL (both provisioned 2026-08-04), so no pulumi-infra change is needed — verified against the repo's secret list.

required: false is deliberate and unchanged in spirit from the URL: passing ${{ secrets.X }} for a secret the repo lacks yields an empty string, which GitHub rejects against a required secret and fails the call before the guard can warn.

Validation

  • zizmor: clean on all four workflows.
  • actionlint: no new findings (only pre-existing self-hosted runner-label noise).
  • actionlint negative control: renaming the secret to CERBERUS_API_TOKENN made it report "secret CERBERUS_API_TOKEN is not defined in ./.github/workflows/cerberus-notify.yml" for all three callers — proving the reusable-workflow call is genuinely resolved, not silently passed.
  • Execution harness: extracted the real run: body and ran it against a stubbed curl. Bearer header is a single argv element; a hostile commit message ("quoted" $(whoami) + backticks, multi-line) is JSON-escaped into commit.title, not executed, first line only; all three missing-secret combinations warn and exit 0 with no curl call; empty commit message is safe under set -u.

The identical fix shipped for stackstate-process-agent in #253 (merged).

VioletCranberry and others added 2 commits August 5, 2026 13:38
Cerberus now requires CI callers to present a shared bearer token. Send it from
the notify script.

Suspend xtrace around the curl and drop --verbose: the script runs under
set -exuo pipefail, so either would echo the token into the job log.
StackVista/cerberus#4 (STAC-24889) adds authentication to the Cerberus
Lambda: every non-Slack request must carry

    Authorization: Bearer <ssm:/cerberus/api-token>

compared constant-time in internal/auth/auth.go verifyBearer. Before it,
the endpoint was completely unauthenticated -- anyone who knew the URL
could lock or unlock a protected branch or post as the Cerberus Slack
bot.

That PR is still open, so the workflow added in STAC-25461 would work as
written today. The moment cerberus#4 deploys, the unauthenticated POST
gets a 401, curl --fail turns the notify job red, and no Slack message
is sent -- exactly the silent-failure condition the job exists to
remove.

Sending the header is forward-compatible: the deployed Lambda ignores
unknown headers, so this is correct either side of that deploy and needs
no coordination.

CERBERUS_API_TOKEN already exists as a repo-level secret alongside
CERBERUS_LAMBDA_URL, so no pulumi-infra change is needed. It is declared
`required: false` for the same reason as the URL -- passing
`${{ secrets.X }}` for a secret the repo lacks yields an empty string,
which GitHub rejects against a *required* secret and fails the call
before the guard can warn. The guard now covers both, so a missing
secret still warns and exits 0 rather than adding a second red job to an
already-failed run.

Also drops `curl --verbose`, which echoes request headers -- the
Authorization header now carries the shared token. GitHub would mask it,
but not emitting it beats relying on masking.

The same fix shipped for stackstate-process-agent as STAC-25529.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
STAC-24889: authenticate Cerberus calls
@LouisParkin

Copy link
Copy Markdown
Author

Removed the whole-line comments from the five workflow YAML files in this PR, per house convention.

Shell comments inside run: blocks are left in place: those are script content rather than YAML commentary. The SHA-pin tag comments and the # zizmor: ignore[...] directives also stay, since both are functional.

The change is behaviour-neutral by construction. I parsed every file before and after the removal and confirmed the loaded YAML documents are structurally identical, and re-ran actionlint and zizmor (no findings).

Where a comment was carrying information a reader genuinely needs, the surrounding YAML already expresses it, for example through name: and description: fields.

LouisParkin and others added 7 commits August 10, 2026 10:19
House convention is that we do not comment YAML files. The ported workflows
carried a large block of explanatory commentary from the GitLab originals.

Only whole-line YAML comments are removed. Shell comments inside run: blocks
are script content and stay, as do the SHA-pin tag comments that make pinned
action versions readable and the zizmor ignore directives, which are both
functional rather than commentary.

Verified by parsing each file before and after: the loaded YAML documents are
structurally identical, so this is a pure comment removal.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Ports the publishing half of pre_release_main_agent_image and
pre_release_cluster_agent_image, plus merge_docker_manifest_main_agent
and merge_docker_manifest_cluster_agent, from GitLab to GitHub Actions.
PR #446 landed the build-and-verify halves; this adds the push.

Each image gets a per-arch publish job that needs: the existing image
build job, and a manifest-merge job that assembles the multi-arch tag
from the two single-arch ones. Gated on `github.event_name == 'push'`
alone: the workflow's push filter only carries the release branch, so
the event check is the whole gate, and a workflow_dispatch on an
arbitrary branch cannot publish. This is the same reasoning the
cerberus-notify gate already uses.

Uses the StackVista/image-pipeline composite actions rather than an
open-coded docker push, matching stackstate-process-agent and the
STAC-24837 direction for product repos. Over publish_image.sh that adds
cosign signatures in both v2 and v3 bundle formats, SBOM and max-mode
provenance attestations, canonical SUSE Observability OCI labels, an
entrypoint ELF-architecture check that catches arch-mismatched images
before they are signed, and refusal to overwrite an existing tag.
Neither Dockerfile declares `ARG BASE_IMAGE`, so base-name is passed
explicitly; both final stages are registry.suse.com/bci/bci-micro.

Tag is the 8-character short SHA, matching GitLab's CI_COMMIT_SHORT_SHA.
The `<branch>-<arch>` tag publish_image.sh also pushed is deliberately
dropped: helm-charts-internal pins the agent and cluster-agent images to
an 8-character SHA (currently 9516cb4, the stackstate-7.78.2 HEAD), and
beest receives the tag as AGENT_HASH_UNDER_TEST, so nothing consumes a
branch-name tag. An org-wide code search for
stackstate-k8s-agent:master, :stackstate-7*, :$CI_COMMIT_REF_SLUG and
the cluster-agent equivalents returns no hits, verified against a
positive control so an empty result is not a false negative.

Both publish jobs need id-token: write for keyless cosign signing, and
are added to each workflow's cerberus-notify needs list so a failed
publish on the release branch still reaches Slack.

Requires STAC-25541 (terraform-infra #75): the stackstate+agent robot
behind QUAY_USER has no write grant on stackstate-k8s-agent or
stackstate-k8s-cluster-agent, since GitLab publishes them with the
legacy gitlabci robot instead. Without it these jobs fail exactly as
process-agent did in STAC-25510.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…roxy BCI bases

Address review feedback on #454.

Gate publication on the image scan: build-agent-image and
build-cluster-agent-image now run image-pipeline's scan-image in `mode: gate`
at the house severity (UNKNOWN..CRITICAL, Grype enabled) directly after the
smoke test. The publish jobs already depend on the build jobs, so a failing
scan now blocks publication instead of only informing. Because
build-agent-image also runs for same-repo pull requests, the gate executes on
this PR's own CI.

Make publication depend on all verification jobs: publish-agent-image now
needs test-deb-renaming as well as build-agent-image, so a .deb that still
carries DataDog branding cannot reach quay.io. test-deb-renaming lives in this
workflow only, so the cluster-agent lane is unaffected.

Pull the BCI bases through the SUSE proxy: both Dockerfiles take
BCI_IMAGE_REGISTRY as a build argument, defaulting to
registry.tooling.stackstate.io/suse/bci. This follows the existing
BASE_IMAGE_REGISTRY convention in the same files, so every build path picks it
up -- the GitHub workflows, the GitLab jobs, and build_images.sh -- without
each caller having to pass a build argument. The apply-oci-labels base-name
input stays on registry.suse.com so the provenance label keeps the upstream
identity that VEX data is keyed on.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tions

The image scan gate added in this branch surfaced a real backlog: 24 findings
in the agent image (embedded CPython 3.13.13, cryptography 48.0.1, two Go
advisories) and 2 in the cluster-agent image. Secret scanning is clean on both.

Keep the gate at mode: gate and declare every finding explicitly instead of
weakening the severity filter, so nothing new can slip in unnoticed. Each
exception carries a <=14-day expiry per the CVE remediation SLA, so the gate
starts failing again if the underlying bumps stall.

The omnibus runtime bumps (CPython 3.13.14, cryptography) are tracked in
STAC-25556; they are software-definition changes, not workflow changes.

LOCAL_IMAGE is renamed to the quay path the publish jobs already use because
the evaluator keys exceptions on the normalised image name, which strips only
the tag or digest and not the registry or namespace.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…itHub

Closes the last two GitLab jobs with no GitHub counterpart: sign_deb and
pre_release_deb (.gitlab-ci-agent.yml lines 604 and 635).

Security prerequisite, not cleanup
----------------------------------
sign_debian_package.sh called printenv unconditionally. On GitLab that dumped
the GPG private key and its passphrase into the job log; stackstate-agent is a
PUBLIC repo, so on GitHub that log is world-readable. It also left the exported
private key in the checkout as gpg_private.key, where any later artifact upload
would collect it. Both are removed before any signing secret is wired in.

Key setup now happens in an ephemeral GNUPGHOME created with mktemp and mode
700, removed by an EXIT trap that also kills the gpg-agent so a preset
passphrase cannot outlive the job on a reused runner. The passphrase reaches
gpg through a mode-600 file inside that directory rather than argv.

Fixes a latent signing bug
--------------------------
The old preset step interpolated an unquoted command substitution into a single
gpg-preset-passphrase call. A key exposes one keygrip per primary and subkey, so
with a signing subkey the second keygrip was passed as a stray argument and
never presetted, leaving signing able to block on a pinentry prompt no CI runner
can answer. Each keygrip is now presetted individually. Verified against a
throwaway key: two keygrips, both presetted.

Shared setup
------------
sign_debian_package.sh and publish_package.sh need the same key in the same
state but are separate processes, and on GitHub may be separate steps, so
neither can rely on a keyring the other left behind. The setup moves into
gpg_signing_setup.sh, sourced by both. gpg-preset-passphrase is auto-detected
across /usr/lib/gnupg2, /usr/lib/gnupg, /usr/libexec and PATH, and the script
fails loudly rather than silently skipping the preset when it is absent.

Both scripts now fall back from CI_PROJECT_DIR to GITHUB_WORKSPACE, and
publish_package.sh falls back from CI_COMMIT_REF_NAME to GITHUB_REF_NAME, so the
apt codename stays the release branch exactly as it was on GitLab.

Workflow
--------
sign-and-publish-deb downloads both architecture artifacts and runs a single
deb-s3 upload. GitLab fanned this out per architecture, so two jobs rewrote the
same apt index concurrently; collecting both first removes that race.

The install script is split in two. generate-install-script runs
inv release.generate-install -t inside the build container, because the task
collection imports python-gitlab and the GitHub and Datadog API helpers and only
loads in the conda environment, and asserts the rendered script contains no
None.s3.amazonaws.com from an unset bucket variable. publish-install-script then
uploads it, so the container job never holds a credential.

Both publishing jobs are gated on push and bound to the agent-pre-release
environment, which carries the deployment branch rule, the signing secrets and
the AWS role from STAC-25545. Pull requests cannot reach them.

Validated: shellcheck -x clean on all three scripts; gpg_signing_setup exercised
against a generated throwaway key covering explicit override, PATH
auto-detection and the missing-binary failure path; actionlint clean apart from
the pre-existing self-hosted runner-label notices; zizmor reports no findings.

Blocked until the agent-pre-release environment, its four SIGNING_* secrets and
AGENT_PRERELEASE_ROLE_ARN exist.

Refs STAC-25546, STAC-25545

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
…tion

Addresses review feedback on PR #455.

[P1] The signing job installed the publisher with `gem install deb-s3`,
resolving the latest code at run time into a job that then executes it with
the package signing key and the pre-release AWS credentials in scope. GitLab
never did this: `sign_deb` ran in a pinned image with deb-s3 already baked in,
so the runtime resolve was a regression introduced by the port.

deb-s3 and its full runtime dependency tree are now pinned in
`.github/deb-s3-gems.sha256` and installed by `.github/scripts/install-deb-s3.sh`,
which fetches each gem at its exact version and verifies it against the SHA256
RubyGems publishes for that release before anything is installed or executed.

The manifest covers nine gems. base64, bigdecimal and logger are deliberately
excluded: aws-sdk-core requires them at ">= 0" and they are Ruby default gems
supplied by the distribution's own ruby package, so pinning them would force a
native build for no supply-chain gain.

The script also links the canonical executable when RubyGems installs a
versioned binstub, since publish_package.sh invokes deb-s3 by bare name, and
ends with a `deb-s3 help` smoke check that activates the whole pinned set so a
missing or incompatible gem fails at install time rather than midway through
publishing.

[P2] publish-install-script depended only on generate-install-script, so it
could overwrite the public install.sh even when the build, signing or apt
upload had failed. GitLab's pre_release_deb required sign_deb. It now requires
both generate-install-script and sign-and-publish-deb, restoring that release
boundary.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
Resolving the rebase conflict against the rewritten base took this branch's
whole copy of build-deb.yml, which predated the exception work on STAC-25457.
That silently reverted two lines, so the gate ran here with no exceptions loaded
and reported all 24 findings as unmanaged.

Restores the quay LOCAL_IMAGE name the evaluator matches exceptions on, and the
exceptions-path input. The workflow diff against the base branch is now
additions only.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin
LouisParkin force-pushed the STAC-25142-agent-lint-unit branch from d7e06f4 to ca28691 Compare August 10, 2026 08:22
STAC-25500 Port the DEB signing and pre-release publishing lane to GitHub Actions
STAC-25457 Publish and sign the agent and cluster-agent images from GitHub Actions
@LouisParkin

Copy link
Copy Markdown
Author

Do not merge yet — the DEB signing credentials are only half-provisioned.

pulumi-infra #283 merged and its apply partially succeeded: stackstate-agent now has the agent-pre-release environment (correctly restricted to protected branches) and the AGENT_PRERELEASE_ROLE_ARN variable, but zero secrets in the environment. The pulumi-infrastructure App lacks the environments permission, so it created the environment and then got 403 writing secrets into it (run 31400655652). Tracked as STAC-25571, approved to fix.

sign-and-publish-deb and publish-install-script are gated if: github.event_name == 'push', so nothing here exercises them and this PR stays green — they would fail on the first push to stackstate-7.78.2 after merge.

Safe to merge once the permission is granted and the pulumi apply is re-run green.

@LouisParkin

Copy link
Copy Markdown
Author

Hold cleared — credentials are provisioned.

`agent-pre-release` environment on this repo now has all four `SIGNING_*` secrets, with a protected-branches-only deployment policy, and `AGENT_PRERELEASE_ROLE_ARN` is set. pulumi-infra #283 is merged and applied (run 31400655652).

Remaining blocker on this PR is review only.

Branch protection required 'Build agent binary (branded / StackState)' and the
cluster-agent equivalent, but the build lanes are an arch matrix, so the emitted
contexts carry a ', amd64' / ', arm64' suffix and those two required contexts
could never report. Every pull request to stackstate-7.78.2 was BLOCKED.

Enumerating the suffixed names would work until the next matrix or rename change
breaks it again, silently. Instead adopt the aggregate already used by
stackstate-agent-integrations: one job per workflow that evaluates its needs and
reports a single stable context.

The aggregate fails on a skipped upstream job rather than passing. Fork pull
requests skip every build and test job, so treating skipped as success would let
a fork merge on a pipeline that ran nothing. mod-tidy is the one documented
exception, skipped when the cache image already covers the module graph.

Publish, signing and manifest jobs are excluded: they are push-only and can
never report on a pull request.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin

Copy link
Copy Markdown
Author

Unblocked — this now needs only a review.

pulumi-infra#286 merged and applied, so the release-branch protection reads:

CI success (lint and unit tests)
CI success (DEB package build)
CI success (binary builds)

All three report success on head 3e2a23fe. The two contexts that could never report (the Build … binary names, stale since the build jobs became an arch matrix) are gone, so BLOCKED is now just the one required approving review — enforce_admins is on, so it cannot be bypassed.

@LouisParkin LouisParkin changed the title STAC-25142 Add GitHub Actions lint + unit-test workflow (agent CI migration phase 1) STAC-25142 Port the agent build, package, and publish pipeline to GitHub Actions Aug 12, 2026
GitLab pipeline schedule 20682 runs the agent pipeline on the release branch
every weekday evening. Archiving the GitLab project without an equivalent here
would end nightly builds silently, so this adds one. Those scheduled runs are
now the only pipelines still executing over there.

That schedule is 22:00 in Europe/Amsterdam. GitHub cron is UTC with no timezone
support, so this uses 20:00 UTC to match during CEST and accepts an hour's drift
in winter, a few minutes off the hour because scheduled runs are delayed at
popular times.

The schedule covers the build, DEB and image workflows but not the unit suites.
Those are two ~2h jobs on self-hosted xlarge runners and they already run on the
pull request and again on the push, so a third nightly run buys no signal.

Publication had to be widened as well. Every publishing job tested for a push
event, and a scheduled run is not one, so the nightly would have built the
packages and images and then discarded them. The Cerberus conditions carried the
same test, which would have left a nightly failure reported to nobody.

Image tags are the commit SHA, so a nightly on an unchanged branch republishes
an identical tag. The point is drift in the base images and fetched
dependencies, not fresh artifacts.

Keying concurrency on the event name stops a push from cancelling a nightly
that is already running.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin
LouisParkin force-pushed the STAC-25142-agent-lint-unit branch from 2a3ce7e to 05d578f Compare August 12, 2026 12:35
The GitHub workflows cover the pipeline, so the GitLab definitions are dead
weight. Only the four StackState-owned files go: the parent pipeline, the agent
child pipeline, the manifest-merge pipeline, and the runner-image pipeline whose
trigger has been commented out for a long time.

Deliberately left in place:

- `.gitlab/`, 129 upstream files that STS pipelines never included but that
  around fifteen upstream `tasks/*.py` modules read. Deleting them would break
  upstream tooling and guarantee conflicts on the next merge.
- `.gitlab-scripts/setup_artifact_registry.sh`, which the GitHub DEB build,
  `tasks/omnibus.py` and an omnibus software definition all call. The name is
  now misleading but the file is live.
- `.ci-builders/`, no longer built by anything but not pipeline definitions.

`tasks/buildimages.py` and two Renovate regex managers read `.gitlab-ci.yml` for
`CI_IMAGE_*` variables. The STS file never defined any, so both were already
no-ops and are left untouched rather than diverging further from upstream.

Documentation is repointed at the workflows. Two claims went stale rather than
moving: the `[cluster-agent]` commit-message optimisation, which neither
pipeline ever implemented, and `AGENT_HELM_CHART_VERSION`, an override the
GitHub lane does not carry.

Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com>
@LouisParkin LouisParkin changed the title STAC-25142 Port the agent build, package, and publish pipeline to GitHub Actions STAC-25142 Move the agent CI to GitHub Actions and retire the GitLab pipelines Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants