Skip to content

WIP terraform engine removal - #6746

Draft
denik wants to merge 12 commits into
mainfrom
denik/remove-terraform-engine
Draft

denik wants to merge 12 commits into
mainfrom
denik/remove-terraform-engine

Conversation

@denik

@denik denik commented Sep 18, 2026

Copy link
Copy Markdown
Contributor

No description provided.

denik and others added 9 commits September 18, 2026 10:13
Reject the terraform engine at selection time (bundle.engine and
DATABRICKS_BUNDLE_ENGINE) with a message pointing at the direct engine or a
CLI v1.17.x downgrade. Delete the terraform engine execution code
(apply/plan/write/interpolate/import/unbind/install/init and tfdyn), keeping
the terraform state-reading and name-mapping utilities still needed by
migration and read-only commands. Simplify deploy/plan/destroy/bind/unbind to
direct-only; a resolved terraform state now errors with guidance to run
"bundle deployment migrate".

Automatic in-place migration of terraform state is reintroduced separately.
CI acceptance matrix is switched to the direct variant only.

Co-authored-by: Isaac <no-reply@databricks.com>
…v1.18.x

Co-authored-by: Isaac <no-reply@databricks.com>
Terraform, TerraformPlanPath, and TerraformPlanIsEmpty are no longer set or
read after the engine removal.

Co-authored-by: Isaac <no-reply@databricks.com>
Switch the engine matrix to direct only, delete the terraform-variant divergent
golden files, remove the terraform/migration bootstrap tests (they deployed via
the removed terraform engine; migration coverage returns with the in-memory
migration), and regenerate out.test.toml.

Co-authored-by: Isaac <no-reply@databricks.com>
… goldens

Convert the per-test EnvMatrix overrides to direct only, delete the remaining
terraform-only tests (lifecycle/select/unknown-field/direct-only error cases and
terraform-specific config-remote-sync/telemetry paths), drop dead terraform
EnvMatrixExclude entries, and regenerate all goldens. Add the changelog fragment.

Co-authored-by: Isaac <no-reply@databricks.com>
…ructor

Run go mod tidy (removes hashicorp/hc-install and terraform-exec, now only used
by the separate codegen module) and trim their NOTICE entries plus the bundled
terraform-binary entry. Remove the terraform test cell from the CI matrix and
the unreachable deployplan.NewPlanTerraform.

Co-authored-by: Isaac <no-reply@databricks.com>
…tion warning

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
@eng-dev-ecosystem-bot

eng-dev-ecosystem-bot commented Sep 18, 2026

Copy link
Copy Markdown
Collaborator

Integration test report

Commit: e7429f8

Run: 35334331696

Env ✅​pass 🙈​skip Time
✅​ aws linux 357 38 12:33
✅​ aws windows 287 60 10:17
✅​ azure linux 356 38 12:03
✅​ azure windows 286 60 9:59
✅​ gcp linux 357 38 11:51
✅​ gcp windows 287 60 10:57
Top 13 slowest tests (at least 2 minutes):
duration env testname
8:15 aws windows TestAccept
7:35 azure windows TestAccept
7:25 gcp windows TestAccept
6:42 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:41 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:33 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
6:23 gcp linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:19 aws linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=true
6:09 azure linux TestAccept/bundle/config-remote-sync/multiple_resources/DMS=
2:10 gcp windows TestAccept/bundle/debug/fetch-repository-info/DMS=
2:09 gcp windows TestAccept/bundle/debug/fetch-repository-info/DMS=true
2:05 gcp linux TestAccept/bundle/debug/fetch-repository-info/DMS=
2:03 gcp linux TestAccept/bundle/debug/fetch-repository-info/DMS=true

denik and others added 3 commits September 18, 2026 11:47
The engine variable no longer selects anything, so remove it as a test matrix
dimension entirely: drop EnvMatrix/EnvRepl/EnvVaryOutput, rename all .direct.
golden files to plain names, strip $DATABRICKS_BUNDLE_ENGINE from scripts and
_script helpers, and update read_state.py/print_state.py/dbr_runner.py to stop
branching on it. Delete the now-empty test.toml files, the EnvVaryOutput
selftests, and the terraform-vs-direct permissions comparison test. Drop the
engine cell from the CI envfilter.

Restore direct_migration.go (reused by the pre-deploy migration, wired next).

Co-authored-by: Isaac <no-reply@databricks.com>
When the resolved state uses the Terraform engine, convert it to a direct-engine
state in memory (OpenMigratedTerraformState) and open the state DB with it before
planning. The deploy's normal Finalize commits resources.json only if the WAL
records changes; terraform.tfstate (local + remote) is backed up only then. Plan
does not commit. Also fix whitespace flagged by the linter.

Co-authored-by: Isaac <no-reply@databricks.com>
Co-authored-by: Isaac <no-reply@databricks.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants