Skip to content

Api thermal heuristic - #238

Merged
Juliette-Gerbaux merged 130 commits into
mainfrom
api_thermal_heuristic
Aug 11, 2026
Merged

Api thermal heuristic#238
Juliette-Gerbaux merged 130 commits into
mainfrom
api_thermal_heuristic

Conversation

@Juliette-Gerbaux

@Juliette-Gerbaux Juliette-Gerbaux commented Jun 19, 2026

Copy link
Copy Markdown
Contributor

Process ID

Process: GP-01

Description

Adds an integer-strategy field on components, letting a component choose how its model's integer/binary variables are built:

  • exact (default, unchanged behavior) — kept as MILP.
  • relaxed — relaxed to continuous.
  • heuristic (+ heuristic-id) — relaxed to continuous for a first solve, then refined by a built-in heuristic (fast or accurate) that computes tighter bounds from that first solve, followed by an automatic second solve. Triggered transparently by SimulationSession when at least one component uses integer-strategy: heuristic.

Each model declares what its chosen heuristic reads from and writes to via a new models[].heuristics section in optim-config.yml (binding fixed heuristic-element names to the model's own parameter/variable ids and bound types). validate_optim_config() checks heuristic-idheuristics consistency, that every bound id exists on the model with the expected time-dependence, and that heuristic is not combined with resolution.mode: benders-decomposition.

This required a refactor of variable construction in optimization.py: when a model's components are split across relaxed/exact strategy groups, variables are built per-group and then merged into a read-only _MergedGroupVariable view, while a new _linopy_vars_by_component map keeps a reference to each component's real registered linopy Variable so heuristic bound mutations (and solution retrieval, via the new get_component_variable/get_variable_solution) reach the solver correctly instead of silently landing on a detached xr.concat copy.

Impact Analysis

Modules affected: optim_config/ (new HeuristicConfig, IntegerStrategy parsing/validation), study/ (integer_strategy field on ComponentSchema, system.py), simulation/ (optimization.py variable-building refactor, new heuristic_runner.py and thermal_heuristic.py, SimulationSession two-pass solve, simulation_table.py).

Solver output values: Not expected to change for existing configs — integer-strategy defaults to exact, which preserves current variable construction and the single-solve flow. Values only change for components that newly opt in to relaxed or heuristic, which is the intended effect of the feature.

Checklist

  • Unit tests pass (pytest)
  • Type checking passes (mypy)
  • Formatting passes (black, isort)
  • pyproject.toml version bumped if applicable
  • AGENTS.md reviewed for impact and updated if needed

Juliette-Gerbaux and others added 7 commits July 13, 2026 15:20
Cross-check each heuristic input/output declared in optim-config.yml against
the model: the referenced id must exist and have the time-dependence the
fast/accurate thermal heuristics expect (e.g. min_up_duration constant,
generation_power per-timestep), catching mismatches at load time instead of
a runtime crash mid-solve. nb_units_max and cluster_max_generation now
accept either form, with the heuristics broadcasting a scalar internally.
Also warn (instead of silently truncating) when min_up_duration/
min_down_duration resolve to a non-integer number of timesteps.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
@Juliette-Gerbaux
Juliette-Gerbaux marked this pull request as ready for review July 24, 2026 16:53
@Juliette-Gerbaux
Juliette-Gerbaux requested a review from tbittar July 24, 2026 16:54

@tbittar tbittar left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We may flag all e2e heuristic test so that we are able to deactivate them easily from the CI at some point (if they get too long)

Comment thread src/gems_craft/optim_config/parsing.py Outdated
Comment thread src/gems_craft/optim_config/parsing.py Outdated
Comment thread src/gems_craft/optim_config/parsing.py Outdated
Comment thread src/gems_craft/optim_config/parsing.py Outdated
Comment thread src/gems_craft/optim_config/parsing.py Outdated
Comment thread tests/e2e/functional/test_thermal_heuristic_one_cluster.py Outdated
Comment thread tests/e2e/functional/test_thermal_heuristic_one_cluster_with_ramp.py Outdated
Comment thread tests/e2e/functional/test_thermal_heuristic_six_clusters.py Outdated
Comment thread tests/e2e/functional/expected_outputs_three_clusters.py Outdated
Comment thread tests/e2e/functional/test_thermal_heuristic_six_clusters.py Outdated
@tbittar

tbittar commented Jul 31, 2026

Copy link
Copy Markdown
Collaborator

Fix solution retrieval, window size in fast heuristic bugs (b96b782)

Follow-up commit addressing 4 issues surfaced by /code-review on this branch:

1. Root cause: solution reads keyed by an ambiguous merged-variable name
For a model that splits an INTEGER/BINARY variable across relaxed and exact integer_strategy groups, optimization.py registers two real linopy.Variables but merges them into a synthetic xr.concat-ed copy for the general linopy_vars lookup dict. That copy's .name arbitrarily reflects only one of the two real registered names. Anything that read solved values via linopy_model.solution[<name>] (heuristic variable-solution inputs, sequential-mode carry-over extraction, and the simulation table's output/extra-output collectors) would silently get NaN or drop the other group's components.

Fixed by adding OptimizationProblem.get_variable_solution(model_id, var_name), which reads .solution directly off the real per-component-registered variables (_linopy_vars_by_component) and reassembles them — the same mechanism already used for safe bound mutation via get_component_variable. Rewired all affected call sites in heuristic_runner.py, session.py, and simulation_table.py.

2. Heuristics/second solve ran before checking the first solve's status
SimulationSession._run_block applied heuristics and re-solved unconditionally before ever checking whether the first solve reached optimality. Moved the status check to run immediately after the first solve (via a small _check_solved helper), so a failed first solve fails fast instead of running the heuristic step against a non-optimal or missing solution.

3. Fast heuristic crashed on zero minimum durations
find_min_generation_fast computed window_size = max(min_up_duration, min_down_duration), which is 0 for a cluster with no minimum up/down time, and used it as a range() step — raising ValueError: range() arg 3 must not be zero. Clamped to max(..., 1), which degenerates correctly to per-timestep processing.

4. Minor: missing space in the solve-failure error message.

Also updated the FakeProblem test doubles in test_simulation_table_{accessor,export,mock}.py to implement get_variable_solution, matching the new interface.

Verification: full suite green (605 passed, 1 xfailed), mypy clean on all touched files.

Juliette-Gerbaux and others added 9 commits August 5, 2026 11:14
The merged Variable rebuilt for split integer/binary variables was a
detached xr.concat copy: setting .lower/.upper on it silently wrote to
an orphaned copy instead of the solver, and its name/label_range were
inherited from only one of the two groups. Now fails loudly and
carries correct metadata.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Documents the new per-component integer relaxation strategy
(exact/relaxed/heuristic) and the built-in fast/accurate thermal
heuristics in optim-config.md, building.md, AGENTS.md, and the
changelog. Also fixes a mypy error-code annotation in optimization.py.

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Comment thread tests/e2e/functional/optim-config/thermal_heuristic.yml Outdated
Comment thread docs/user-guide/building.md Outdated
for mc in optim_config.models
for heuristic_config in (mc.heuristic or [])
}
heuristic_config_map = get_heuristic_config_map(optim_config)

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is weird to parse the optim config file into the runner. why not enriching the Optimconfig class with the heuristic config, then simply use the optim config object (or its heuristics attribute here) ?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will be treated in another PR.

Comment thread tests/unittests/gems_runner/simulation/test_simulation_table_mock.py Outdated
@tbittar

tbittar commented Aug 10, 2026

Copy link
Copy Markdown
Collaborator

It seems some problems are too large to use Xpress in CI... use highs instead

@Juliette-Gerbaux
Juliette-Gerbaux merged commit cfd0309 into main Aug 11, 2026
2 checks passed
@Juliette-Gerbaux
Juliette-Gerbaux deleted the api_thermal_heuristic branch August 11, 2026 12:08
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants