feat: run comparison - pipeline diff engine - #2596
Draft
camielvs wants to merge 1 commit into
Draft
Conversation
🎩 PreviewA preview build has been created at: |
Collaborator
Author
This was referenced Jul 30, 2026
There was a problem hiding this comment.
Pull request overview
Adds the core “pipeline diff” utilities intended for a Run Comparison view, including per-task diffing (component version/args/annotations/cache) plus input/output alignment and outcome metadata.
Changes:
- Added a pipeline comparison/diff engine (
buildPipelineComparison) with task/input/output diff outputs and summary counts. - Added vitest coverage for the pipeline diff engine.
- Added a shared helper to aggregate per-task execution statuses, plus a CompareView hook to load per-side run data.
Reviewed changes
Copilot reviewed 4 out of 4 changed files in this pull request and generated 5 comments.
| File | Description |
|---|---|
| src/utils/executionStatus.ts | Adds buildTaskExecutionStatusMap to aggregate per-task statuses from run details/state. |
| src/routes/v2/pages/CompareView/utils/comparePipelines.ts | Introduces the run-to-run pipeline diff engine and supporting diff helpers/types. |
| src/routes/v2/pages/CompareView/utils/comparePipelines.test.ts | Adds unit tests covering task/input/output diff scenarios and counts/outcome flags. |
| src/routes/v2/pages/CompareView/hooks/useRunComparisonSide.ts | Adds a hook to load one side of a comparison (spec, per-task status, per-task execution ids, metadata). |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Comment on lines
+175
to
+183
| /** | ||
| * Build a map of task id → aggregated execution status by joining a run's | ||
| * task→execution id mapping against its per-execution status stats. | ||
| */ | ||
| export function buildTaskExecutionStatusMap( | ||
| details?: GetExecutionInfoResponse, | ||
| state?: GetGraphExecutionStateResponse, | ||
| ): Map<string, string> { | ||
| const taskExecutionStatusMap = new Map<string, string>(); |
Comment on lines
+179
to
+201
| export function buildTaskExecutionStatusMap( | ||
| details?: GetExecutionInfoResponse, | ||
| state?: GetGraphExecutionStateResponse, | ||
| ): Map<string, string> { | ||
| const taskExecutionStatusMap = new Map<string, string>(); | ||
|
|
||
| if (!details?.child_task_execution_ids) { | ||
| return taskExecutionStatusMap; | ||
| } | ||
|
|
||
| for (const [taskId, executionId] of Object.entries( | ||
| details.child_task_execution_ids, | ||
| )) { | ||
| const statusStats = state?.child_execution_status_stats?.[executionId]; | ||
| const aggregated = getOverallExecutionStatusFromStats(statusStats); | ||
|
|
||
| if (aggregated) { | ||
| taskExecutionStatusMap.set(taskId, aggregated); | ||
| } | ||
| } | ||
|
|
||
| return taskExecutionStatusMap; | ||
| } |
Comment on lines
+1
to
+4
| import { usePipelineRunData } from "@/hooks/usePipelineRunData"; | ||
| import { useFetchPipelineRunMetadata } from "@/services/executionService"; | ||
| import type { ComponentSpec } from "@/utils/componentSpec"; | ||
| import { buildTaskExecutionStatusMap } from "@/utils/executionStatus"; |
Comment on lines
+32
to
+39
| const spec = details?.task_spec.componentRef.spec as | ||
| ComponentSpec | undefined; | ||
|
|
||
| const taskStatusMap = buildTaskExecutionStatusMap(details, state); | ||
|
|
||
| const taskExecutionIdMap = new Map<string, string>( | ||
| Object.entries(details?.child_task_execution_ids ?? {}), | ||
| ); |
Comment on lines
+25
to
+27
| export function useRunComparisonSide(runId: string): RunComparisonSide { | ||
| const { executionData, isLoading, error } = usePipelineRunData(runId); | ||
| const { data: runMetadata } = useFetchPipelineRunMetadata(runId || undefined); |
camielvs
force-pushed
the
cmp-01-diff-engine
branch
from
July 31, 2026 00:24
8c0a61b to
2d2c65a
Compare
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 5 out of 5 changed files in this pull request and generated no new comments.
Comments suppressed due to low confidence (3)
src/routes/v2/pages/CompareView/hooks/useRunComparisonSide.ts:33
componentRef.specfrom the API isComponentSpecOutput | null | undefined, but this cast can allow a runtimenullto flow through asComponentSpec | undefined. Normalizingnulltoundefinedkeeps the runtime value aligned with the declared type and avoids surprising consumers.
const spec = details?.task_spec.componentRef.spec as
ComponentSpec | undefined;
src/routes/v2/pages/CompareView/utils/comparePipelines.ts:194
sameComponentVersioncan befalseeven when the task is treated as unchanged: if bothcomponentRef.digestvalues are missing but thecomponentRefobjects are equal,isComponentChanged(a, b)returns false (so the task is unchanged) whilesameComponentVersionstays false. This creates an inconsistent/incorrect signal for downstream UI (e.g. summaries).
digestA,
digestB,
sameComponentVersion: Boolean(digestA && digestB && digestA === digestB),
statusA,
src/routes/v2/pages/CompareView/utils/comparePipelines.ts:202
outcomeChangedcurrently compares status strings even when the task only exists on one side (added/removed). If the present side has a status and the missing side isundefined, this will markoutcomeChangedand incrementcounts.outcomeChanged, which doesn’t match the intent of “same task, different outcome”.
cacheDisabledB,
cacheChanged,
outcomeChanged: (statusA ?? "") !== (statusB ?? ""),
argumentDiffs,
This was referenced Jul 31, 2026
camielvs
force-pushed
the
cmp-01-diff-engine
branch
3 times, most recently
from
August 14, 2026 21:01
160bb74 to
cfcc893
Compare
3 tasks
camielvs
force-pushed
the
cmp-01-diff-engine
branch
from
August 14, 2026 23:47
cfcc893 to
186b9c5
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
First PR in the Compare Runs stack. Adds the core engine that diffs two pipeline runs — no UI yet, just the logic everything else builds on.
Given two runs it aligns their tasks by id and works out, for each one, whether it was added, removed, changed, or unchanged. "Changed" covers a different component version, different arguments, different annotations, or a flipped cache setting. Pipeline inputs and outputs are compared the same way (including detecting when an output was rewired to a different task). Layout-only editor annotations (node positions, colours, etc.) are stripped out first so they don't show up as noise.
It also tracks per-task execution outcome, so a task whose spec is identical but that succeeded in one run and failed in the other is still flagged as "outcome differs".
A small helper for loading a single run's spec and execution status (
useRunComparisonSide) and a sharedbuildTaskExecutionStatusMaputil round out the PR.Related Issue and Pull requests
Part of the Compare Runs stack. Branches off
master; the next PR (#2597) builds on this branch.Type of Change
Checklist
Screenshots (if applicable)
Test Instructions
This PR is pure logic and ships behind the (not-yet-added)
compare-runsflag, so there's nothing to click through on its own.npm run test -- comparePipelinesTo exercise it in the real UI, check out the top of the stack (#2603), enable the Compare runs beta flag, and compare two runs.
Additional Comments