fix: remove USING joins from health score copy tree (CM-XXXX)

[epipav]

Problem

The nightly health_score_copy COPY pipe (00:50 UTC) fails with:

BAD_ARGUMENTS: JOIN LEFT JOIN ... USING (segmentId) USING clause expected identifier. Actual segmentId.

ClickHouse 25.3+ is stricter about JOIN ... USING when the joined sides are subqueries with aliased columns — which every health_score_* metric pipe is (aggregated GROUP BY subqueries). Since Tinybird inlines all referenced pipes into one query at copy time, the whole query tree is affected, not just the top-level pipe.

Change

Convert every USING (key) join in the health_score_copy query tree to an explicit ON left.key = right.key, preserving join type and keys:

• health_score_copy.pipe — 13 joins in health_score_copy_data (11× segmentId, 2× id)
• health_score_retention.pipe — 1 join (ON cur.segmentId = prev.segmentId)
• health_score_security.pipe — 1 inner join on repo; the select-list repo is qualified as security_deduplicated_merged_ds.repo AS repo since both sides keep the column after an ON join

Output columns (names, order), join semantics, and the health_score_copy_ds target schema are unchanged.

Validation

• On tinybird-local (ClickHouse 25.8.23.6) with the full dependency closure pushed and insightsProjects fixtures: the rewritten 13-join node returns byte-identical results to the original (38 rows, matching hash), and the fixed retention/security pipes push and execute cleanly.
• The production BAD_ARGUMENTS originates in Tinybird cloud's copy-job execution path and does not reproduce through the local classic image; the rewrite follows Tinybird's recommended remediation.
• tb fmt --diff clean on all three files (tinybird-cli 5.x, matching CI).

Deferred (follow-ups)

9 other pipes still contain the same USING construct and will likely need the same conversion as they hit the stricter path: repo_health_score_copy, repo_health_score_retention, project_insights_copy, insights_projects_populated_copy, org_dash_metric_copy_pipe, collections_oss_index, categories_oss_index, category_groups_oss_index, activity_heatmap_by_weekday_and_2hours_blocks. Deliberately excluded here to keep this fix minimal.

[cursor]

PR Summary

Low Risk
SQL-only join syntax change with preserved keys and validated byte-identical results on the main copy node; no app or auth changes.

Overview
Fixes nightly health_score_copy COPY job failures on ClickHouse 25.3+ (BAD_ARGUMENTS on JOIN ... USING when metric pipes inline as aggregated subqueries).

health_score_copy.pipe: All 13 LEFT JOIN ... USING (segmentId|id) joins in health_score_copy_data are rewritten to explicit ON health_score_copy_project_repo_deduplicated.<key> = <metric_pipe>.<key>.

health_score_retention.pipe: The current/previous quarter join uses ON cur.segmentId = prev.segmentId instead of USING (segmentId).

health_score_security.pipe: The security/repo join uses ON security_deduplicated_merged_ds.repo = health_score_security_project_repos.repo, with repo in the select list qualified as security_deduplicated_merged_ds.repo AS repo so the output schema stays the same.

Join types, keys, column names/order, and downstream health_score_copy_ds semantics are unchanged; this is a query-shape fix for stricter ClickHouse/Tinybird copy inlining.

^{Reviewed by Cursor Bugbot for commit 349d233. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

⚠️ Jira Issue Key Missing

Your PR title doesn't contain a Jira issue key. Consider adding it for better traceability.

Example:

• feat: add user authentication (CM-123)
• feat: add user authentication (IN-123)

Projects:

• CM: Community Data Platform
• IN: Insights

Please add a Jira issue key to your PR title.

[Copilot]

Pull request overview

Replaces incompatible Tinybird USING joins with explicit ON conditions in the health-score COPY dependency tree.

Changes:

• Rewrites 13 health-score aggregation joins.
• Updates retention and security joins while preserving output columns.
• Required: Replace CM-XXXX in the PR title with a numeric JIRA key.

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated no comments.

File	Description
health_score_copy.pipe	Rewrites metric joins.
health_score_retention.pipe	Rewrites the quarter join.
health_score_security.pipe	Rewrites and qualifies the repository join.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.