[branch-54] fix: don't duplicate volatile expressions when pushing projection into file scan (backport #23395, adapted)

[fordN]

Which issue does this PR close?

• Backport of fix: don't duplicate volatile expressions when pushing projection into file scan #23395 to branch-54 (Release DataFusion 54.1.0 (minor/patch) Release #22547)
• Closes Projection pushdown into file scan duplicates non-deterministic functions (regression in 52.0.0) #23220.

Rationale for this change

The reported bug showed that when volatile (non-deterministic) expressions are referenced multiple times in an outer function the query results can be incorrect.

To produce correct results, a volatile expression (e.g. random(), uuid()) aliased once in a subquery and referenced multiple times must be evaluated once and reused. Since 52.0.0 this evaluation pattern for volatile functions has been broken; the physical projection-pushdown rule merges the outer projection into the file DataSourceExec, inlining the aliased volatile expression at each reference site. Instead of being evaluated once, the volatile expression is evaluated N times, so the references diverge:

SELECT s.r AS x, s.r AS y FROM (SELECT random() AS r FROM t) AS s;  -- x != y on >= 52.0.0

This was correct in 51.0.0 and regressed in 52.0.0/53.0.0. It reproduces on file scans (Parquet/CSV) but not in-memory tables, and was surfaced downstream in Ibis. Worth noting, #10337 appears to report the same class of bug but a different cause (I'll try to look into that issue soon as well).

What changes are included in this PR?

FileScanConfig::try_swapping_with_projection now declines to merge a projection into the file source when the merge would inline a volatile expression that the incoming projection references. The check reuses the existing is_volatile() utility from datafusion_physical_expr_common — the same volatility gate the physical ProjectionPushdown and FilterPushdown rules already apply — so file-source projection merging is now consistent with them. Deterministic expressions still merge freely.

The guard blocks when a volatile inner expression is referenced at least once (not only more than once), because a single outer expression can itself duplicate the value (e.g.r + r).

Are these changes tested?

Yes:

• Unit tests in file_scan_config.rs:
  • volatile referenced ≥1× → blocked;
  • deterministic computed and column-only → allowed;
  • unreferenced volatile → allowed;
  • single-expression self-reference (r + r) → blocked;
  • volatile nested in arithmetic → blocked;
  • empty → allowed.
• A regression test in projection_pushdown.slt asserting x = y for the aliased-random() pattern over a Parquet scan.
• Full datafusion-sqllogictest suite passes.

Are there any user-facing changes?

No API changes! Query results for the affected pattern are corrected (a volatile expression aliased in a subquery is no longer duplicated by projection pushdown into a file scan). Physical plans for such queries now retain a ProjectionExec above the scan rather than inlining the volatile expression into the DataSourceExec projection.

[alamb]

Thanks @fordN -- this looks good to me. I had an agent confirm it is a clean backport from main

The audit failures I don't think are relate to this PR -- instead it due to changing Cargo.toml and triggering audit checks. I made a separate PR to fix this:

• #23607