[POC - do not merge] TT-18139: release pipelines as reusable workflows

[konrad-sol]

Do not merge. POC for TT-18139.

Moves the gromit-generated release pipelines into reusable workflows here, so a
pipeline change stops fanning out to ~24 repo/branch combinations.

What this adds

File	Serves
release-tyk-analytics.yml + 2 nested	tyk-analytics
release-tyk.yml + 1 nested	tyk
release-common.yml	tyk-pump, tyk-sink, portal, tyk-identity-broker
.github/actions/ecr-login	moved from tyk; generic

Four entry points, one per pipeline shape. tyk and tyk-analytics need their own
because each has a repo-local release-tests.yml; the other four differ only by
data plus two flags (build_fips, run_api_tests).

Caller side

release.yml in each product repo becomes ~30 lines:

Repo	Before	After	PR
tyk-analytics	927	31	TykTechnologies/tyk-analytics#6176 ✅ green
tyk	1291	31	TykTechnologies/tyk#8682 ✅ green
portal	602	28	TykTechnologies/portal#1994 ✅ green

Verbatim ports

Each entry point is a line-for-line copy of the generated release.yml, with
only three edits: on: triggers become workflow_call, the single-leg matrix
becomes a golang_cross input plus constant env, and the dead publish-leg gate
is dropped. release-common.yml additionally parameterises the data axes.

Findings from the POC

• OIDC works unchanged — the sub claim resolves to the caller, so no AWS
  trust-policy changes.
• vars and actions/checkout resolve to the caller, so per-repo runners
  and ci/ files keep working.
• A single failed job inside a called workflow can be re-run on its own —
  granularity is preserved.
• ./ means different things for workflows and actions.
  ./.github/workflows/x.yml resolves against the repo owning the workflow;
  ./.github/actions/x resolves against the checked-out workspace. That broke
  ecr-login until it was referenced repo-qualified.
• Re-running a run pins the reusable workflow version resolved at creation,
  so a re-run will not pick up a hotfix.
• Caller workflow-level env: does not propagate, and env is unavailable
  in a calling job's with:.
• drift-check needs rethinking — it compares against a full gromit render,
  so a thin caller legitimately fails it.
• ai-studio is out of scope — it left gromit in Oct 2025 and is hand-maintained,
  with a component/edition matrix gromit cannot express. Its two config entries
  (ai-studio and midsommar, the same repo via rename) render files nobody uses.

Not done

gromit templates still generate the old full files; emitting thin callers is the
follow-up. Required status checks become release / aggregator-ci-test and need
renaming per repo.

🤖 Generated with Claude Code

[github-actions]

zizmor findings

Severity	Count
High	126
Medium	153
Low	7
Info	32

Full details are in the workflow run.

[probelabs]

This Proof of Concept (POC) refactors the release pipelines for multiple Tyk products into centralized, reusable GitHub workflows. The primary goal is to eliminate widespread duplication of release logic across numerous repositories, simplifying maintenance by making this github-actions repository the single source of truth. Product repositories will replace their large, generated release.yml files with a small (~30 line) caller that invokes these new reusable workflows.

Files Changed Analysis

This PR adds 7 new files, totaling 3,238 lines of code, with no deletions. The changes establish a new, centralized framework for release pipelines.

• Reusable Workflows: The core of the PR consists of three main reusable workflows triggered by workflow_call:

  • .github/workflows/release-common.yml: A generic, parameterized workflow for products like tyk-pump, tyk-sink, and portal.
  • .github/workflows/release-tyk.yml: A specialized workflow for the tyk gateway, which includes complex logic for building multiple image variants (EE, FIPS, std) and dynamically resolving the correct tyk-analytics image for integration testing.
  • .github/workflows/release-tyk-analytics.yml: A dedicated workflow for tyk-analytics, handling specific steps for its UI assets.

• Nested Test Workflows: To improve modularity, the main workflows call smaller, dedicated workflows for running tests, such as release-tyk-release-tests.yml and release-tyk-analytics-release-tests.yml.

• Composite Action: A new composite action, .github/actions/ecr-login/action.yml, is introduced to encapsulate and reuse the AWS ECR login process.

Architecture & Impact Assessment

• What this PR accomplishes: This PR shifts the release engineering strategy from a decentralized model, where large workflow files are generated into each repository, to a centralized, service-oriented model. This repository becomes the single source of truth for release logic.

• Key technical changes introduced:

  • Extensive use of workflow_call to create reusable and parameterizable pipelines.
  • Centralization of complex build, test, and deployment logic, including Docker image creation, multi-platform builds, and publishing to various registries.
  • Introduction of dynamic logic in release-tyk.yml to resolve the appropriate dashboard image for testing based on the PR's context.

• Affected system components:

  • Product Repositories: Will require their release.yml files to be replaced with a small caller workflow.
  • Gromit Tool: The internal tool for generating release pipelines will need to be updated to emit the new, thin callers.
  • Drift-check: The existing mechanism for detecting configuration drift is incompatible with this model and will need to be re-evaluated.
  • Branch Protection: Required status checks in GitHub will need to be updated to reflect the new, centralized job names.

• Component Interaction Diagram:

graph TD
    subgraph "Product Repositories (Callers)"
        A[tyk] --> W_TYK
        B[tyk-analytics] --> W_TA
        C[tyk-pump, tyk-sink, etc.] --> W_COMMON
    end

    subgraph "github-actions Repo (Reusable Workflows)"
        W_TYK(release-tyk.yml)
        W_TA(release-tyk-analytics.yml)
        W_COMMON(release-common.yml)
    end

    W_TYK --> T_TYK(release-tyk-release-tests.yml)
    W_TA --> T_TA(release-tyk-analytics-release-tests.yml)
    W_TYK & W_TA & W_COMMON --> A_ECR(ecr-login action)

Loading

Scope Discovery & Context Expansion

This PR represents a fundamental modernization of the CI/CD and release engineering strategy. The impact extends to developer workflows, release processes, and internal tooling.

To fully understand the context, the next steps would be to investigate:

• Gromit Codebase: Examine the templates used by the gromit tool to understand what changes are needed to produce the new, smaller caller files.
• Example Product Repository: Inspect the main branch of a repository like TykTechnologies/tyk to see the "before" state of the release.yml file and its branch protection rules.
• Drift-Check Implementation: Find the code for the drift-check workflow to understand why it's incompatible and how it could be adapted.

Metadata

• Review Effort: 5 / 5
• Primary Label: n/a

---

Powered by Visor from Probelabs

Last updated: 2026-09-09T09:48:28.490Z | Triggered by: pr_updated | Commit: b7a8854

💡 TIP: You can chat with Visor using /visor ask <your question>

[probelabs]

Security Issues (12)

Severity	Location	Issue
🔴 Critical	.github/workflows/release-common.yml:255	The `docker run` command uses the `--privileged` flag, which disables container security mechanisms and grants the container full access to the host machine. A compromise of the container could lead to a compromise of the CI runner, which is a form of Remote Code Execution on the infrastructure. 💡 Suggestion Avoid using the `--privileged` flag. For building Docker images within a CI job, consider using tools like Kaniko or Buildah which do not require privileged access. If Docker-in-Docker is required, explore rootless Docker options.
🔴 Critical	.github/workflows/release-tyk-analytics.yml:238	The `docker run` command uses the `--privileged` flag, which disables container security mechanisms and grants the container full access to the host machine. A compromise of the container could lead to a compromise of the CI runner, which is a form of Remote Code Execution on the infrastructure. 💡 Suggestion Avoid using the `--privileged` flag. For building Docker images within a CI job, consider using tools like Kaniko or Buildah which do not require privileged access. If Docker-in-Docker is required, explore rootless Docker options.
🔴 Critical	.github/workflows/release-tyk.yml:191	The `docker run` command uses the `--privileged` flag, which disables container security mechanisms and grants the container full access to the host machine. A compromise of the container could lead to a compromise of the CI runner, which is a form of Remote Code Execution on the infrastructure. 💡 Suggestion Avoid using the `--privileged` flag. For building Docker images within a CI job, consider using tools like Kaniko or Buildah which do not require privileged access. If Docker-in-Docker is required, explore rootless Docker options.
🔴 Critical	.github/workflows/release-tyk.yml:989	The `docker run` command inside the build script uses the `--privileged` flag, which disables container security mechanisms and grants the container full access to the host machine. A compromise of the container could lead to a compromise of the CI runner, which is a form of Remote Code Execution on the infrastructure. 💡 Suggestion Avoid using the `--privileged` flag. For building Docker images within a CI job, consider using tools like Kaniko or Buildah which do not require privileged access. If Docker-in-Docker is required, explore rootless Docker options.
🟠 Error	.github/workflows/release-common.yml:220	The workflow input `golang_cross` is used directly in a shell command to specify a Docker image tag. An untrusted input could contain shell metacharacters and lead to arbitrary command injection. For example, an input of `latest; rm -rf /` would be executed. 💡 Suggestion Validate the `golang_cross` input against a strict allow-list or regex to ensure it only contains characters valid for a Docker image tag (e.g., `^[a-zA-Z0-9._-]+$`).
🟠 Error	.github/workflows/release-common.yml:255	The workflow input `repo_name` is used to construct a path for a volume mount (`-v ...:/go/src/github.com/TykTechnologies/${{ inputs.repo_name }}`). A malicious input containing `../` could allow mounting directories outside the intended workspace, potentially leading to unexpected behavior or information disclosure within the build container. 💡 Suggestion Validate the `repo_name` input to ensure it only contains characters valid for a repository name and does not contain `.` or `/`. A regex such as `^[a-zA-Z0-9_-]+$` would be appropriate.
🟠 Error	.github/workflows/release-tyk-analytics-swagger.yml:83	The workflow downloads and executes the `venom` binary without verifying its checksum. This creates a supply chain risk, as a compromised binary could be executed if the download source is compromised. A `TODO` comment in the code acknowledges this vulnerability. 💡 Suggestion Download the checksum file for the release and verify the SHA256 hash of the downloaded binary before making it executable. This is done for `docker-scout-cli` in other workflows and the same pattern should be applied here.
🟠 Error	.github/workflows/release-tyk-analytics.yml:190	The workflow input `golang_cross` is used directly in a shell command to specify a Docker image tag. An untrusted input could contain shell metacharacters and lead to arbitrary command injection. For example, an input of `latest; rm -rf /` would be executed. 💡 Suggestion Validate the `golang_cross` input against a strict allow-list or regex to ensure it only contains characters valid for a Docker image tag (e.g., `^[a-zA-Z0-9._-]+$`).
🟠 Error	.github/workflows/release-tyk.yml:143	The workflow input `golang_cross` is used directly in a shell command to specify a Docker image tag. An untrusted input could contain shell metacharacters and lead to arbitrary command injection. For example, an input of `latest; rm -rf /` would be executed. 💡 Suggestion Validate the `golang_cross` input against a strict allow-list or regex to ensure it only contains characters valid for a Docker image tag (e.g., `^[a-zA-Z0-9._-]+$`).
🟡 Warning	.github/workflows/release-common.yml:256	The Docker socket is mounted into the build container, allowing processes inside the container to control the host's Docker daemon. This can be abused to break out of the container and gain full control over the CI runner. The risk is elevated as this container also runs with `--privileged`. 💡 Suggestion Avoid mounting the Docker socket. For building images, prefer tools like Kaniko or Buildah that operate without a Docker daemon. This change would also help in removing the need for the `--privileged` flag.
🟡 Warning	.github/workflows/release-tyk-analytics.yml:249	The Docker socket is mounted into the build container, allowing processes inside the container to control the host's Docker daemon. This can be abused to break out of the container and gain full control over the CI runner. The risk is elevated as this container also runs with `--privileged`. 💡 Suggestion Avoid mounting the Docker socket. For building images, prefer tools like Kaniko or Buildah that operate without a Docker daemon. This change would also help in removing the need for the `--privileged` flag.
🟡 Warning	.github/workflows/release-tyk.yml:202	The Docker socket is mounted into the build container, allowing processes inside the container to control the host's Docker daemon. This can be abused to break out of the container and gain full control over the CI runner. The risk is elevated as this container also runs with `--privileged`. 💡 Suggestion Avoid mounting the Docker socket. For building images, prefer tools like Kaniko or Buildah that operate without a Docker daemon. This change would also help in removing the need for the `--privileged` flag.

Architecture Issues (2)

Severity	Location	Issue
🟠 Error	.github/workflows/release-common.yml:338-660	The core logic for building, pushing, and verifying Docker images is duplicated across the three main reusable workflows (`release-common.yml`, `release-tyk.yml`, `release-tyk-analytics.yml`). This includes multiple, near-identical calls to `docker/build-push-action` and a large, repeated `verify_manifest` shell script. For example, the `verify_manifest` script is duplicated 14 times across the three files. This undermines the goal of centralization and will make future updates to the image publishing process error-prone and tedious. 💡 Suggestion Refactor the Docker build, push, and verification logic into a new, dedicated reusable workflow (e.g., `reusable-docker-publish.yml`). This new workflow would accept parameters like image name, platforms, and build arguments. The main workflows would then call this reusable workflow, replacing hundreds of lines of duplicated code with a few `uses:` blocks. The `verify_manifest` script should be extracted into a composite action that is called by the new workflow.
🟡 Warning	.github/workflows/release-tyk.yml:904-1017	The `resolve-dashboard-image` job contains a complex, 100+ line shell script embedded directly within the workflow YAML. This script implements critical business logic for determining which version of a dependency to use for testing. Embedding this logic in YAML makes it difficult to read, maintain, debug, and test. A similar issue exists for the multi-line build scripts in all three main workflows. 💡 Suggestion For complex logic like the dashboard image resolution, create a custom GitHub Action (e.g., using JavaScript/TypeScript). This allows the logic to be properly unit-tested and maintained as code. For simpler multi-line scripts like the build commands, move them to separate script files (e.g., `scripts/build.sh`) within this repository and have the workflow step execute that script. This improves readability and separation of concerns.

Performance Issues (2)

Severity	Location	Issue
🟡 Warning	.github/workflows/release-tyk-analytics-release-tests.yml:25-38	Smoke tests are executed serially within a single job by looping over subdirectories in `smoke-tests/*/`. This can become a performance bottleneck as the number of test suites or their execution time increases. Each test suite runs sequentially, leading to a total execution time that is the sum of all individual test times. 💡 Suggestion To improve performance, parallelize the test execution. Refactor the `smoke-tests` job to use a `strategy: matrix` to run each test suite in a separate, parallel job. This requires a preliminary job to identify the test directories and output them as a JSON array for the matrix.
🟡 Warning	.github/workflows/release-tyk-release-tests.yml:87-128	Tests in `/ci/tests/` and `/ci/smoke-tests/` are executed serially within single jobs by looping over subdirectories (lines 91 and 121). This can become a performance bottleneck as the number of test suites or their execution time increases. Each test suite runs sequentially, leading to a total execution time that is the sum of all individual test times. 💡 Suggestion To improve performance, parallelize the test execution. Refactor the `ci-tests` and `smoke-tests` jobs to use a `strategy: matrix` to run each test suite in a separate, parallel job. This requires a preliminary job to identify the test directories and output them as a JSON array for the matrix.

Quality Issues (6)

Severity	Location	Issue
🔴 Critical	.github/workflows/release-tyk-analytics-swagger.yml:87	The workflow downloads the `venom` binary using `curl` and executes it without verifying its integrity via a checksum. A comment 'TODO(security): verify checksum' acknowledges the risk. This is a supply-chain vulnerability; if the `venom` release asset were compromised, this workflow would execute malicious code. 💡 Suggestion Add a step to download the checksum file from the release page and verify the downloaded binary's hash before making it executable. The installation of `docker-scout` in other workflows provides a good example of how to do this correctly.
🟠 Error	.github/workflows/release-common.yml:90-645	The `goreleaser` job, which handles building binaries, packaging, and pushing Docker images, is almost identically replicated across `.github/workflows/release-common.yml`, `.github/workflows/release-tyk.yml`, and `.github/workflows/release-tyk-analytics.yml`. This introduces over 500 lines of duplicated code in each file, severely impacting maintainability and violating the DRY principle. A change to the build logic would need to be manually synchronized across all three files. 💡 Suggestion Refactor the common logic from the `goreleaser` job into a new, separate reusable workflow (e.g., `build-and-publish.yml`). This new workflow would accept inputs for the few values that differ between the pipelines (like package names, Docker repo names, etc.). The parent workflows would then become much simpler, responsible only for setting up specific prerequisites and calling the shared build workflow with the correct parameters.
🟠 Error	.github/workflows/release-tyk.yml:914-1025	The `resolve-dashboard-image` job contains a complex, 100+ line shell script embedded directly in the workflow YAML. This script includes conditional logic, API calls (`aws`, `git`), and determines the strategy for which dashboard image to use. Embedding complex logic in YAML makes it very difficult to read, maintain, debug, and test. 💡 Suggestion Extract this script into a separate, executable file within the repository (e.g., `scripts/resolve-dashboard-image.sh`). The workflow step can then simply call this script. This improves separation of concerns, makes the script independently testable, and keeps the workflow file focused on orchestration rather than implementation.
🟡 Warning	.github/workflows/release-common.yml:350-395	The shell script logic to verify that Docker image layers are gzip-compressed is duplicated four times within this workflow, and is also present in `release-tyk.yml` and `release-tyk-analytics.yml`. This violates the DRY principle and makes the script harder to update. 💡 Suggestion Extract the verification script into a dedicated composite action (e.g., `.github/actions/verify-image-compression/action.yml`). The action would take the image tag as an input. This would replace ~45 lines of duplicated script with a single `uses:` line in each location.
🟡 Warning	.github/workflows/release-tyk-analytics-release-tests.yml:64	The `gh-logs-analyser` action is referenced using a commit SHA that points to the `main` branch. This means the workflow will always use the version of the action from the tip of `main`, not the version from the commit being tested. This can lead to non-hermetic builds and unexpected failures if a breaking change is merged to `main`. 💡 Suggestion To ensure workflows are hermetic, reference the action locally using `uses: ./.github/actions/gh-logs-analyser`. This guarantees that the version of the action being executed is the one from the same commit as the workflow file.
🟡 Warning	.github/workflows/release-tyk-release-tests.yml:20	The `ecr-login` action is referenced using the feature branch name `@TT-18139-releng-poc`. This is acceptable for a proof-of-concept, but it must be updated before merging. Relying on a feature branch means the workflow will break if the branch is deleted. 💡 Suggestion Before merging, update this reference to a stable and long-lived reference. The preferred method would be to use a local path reference `uses: ./.github/actions/ecr-login` to ensure the action version matches the workflow version.

---

Powered by Visor from Probelabs

Last updated: 2026-09-09T09:48:07.758Z | Triggered by: pr_updated | Commit: b7a8854

💡 TIP: You can chat with Visor using /visor ask <your question>