Skip to content

Chore/small improvements - #2200

Open
Raghavan1988 wants to merge 7 commits into
NovaSky-AI:mainfrom
Raghavan1988:chore/small-improvements
Open

Chore/small improvements#2200
Raghavan1988 wants to merge 7 commits into
NovaSky-AI:mainfrom
Raghavan1988:chore/small-improvements

Conversation

@Raghavan1988

@Raghavan1988 Raghavan1988 commented Sep 12, 2026

Copy link
Copy Markdown

Note

Low Risk
Changes are mostly validation, typing, and DRY refactors; the Env.init **kwargs change is a minor signature fix with existing subclasses using named parameters.

Overview
This PR tightens configuration and API ergonomics with clearer validation and small refactors, plus clearer GSM8K test documentation.

Validation: _import_object in the agent auto-loader now rejects import paths without a dot with an explicit error. load_env_creator rejects malformed registry entry points unless they contain exactly one :.

API / behavior: The base Env.init signature is updated from *kwargs to **kwargs. Metric aggregation in default_aggregate_metrics is simplified to average any int/float (including bools) and skip other types. check_is_vlm uses a single getattr(..., None) check.

Refactor: Boolean SkyRL env vars (SKYRL_LD_LIBRARY_PATH_EXPORT, SKYRL_PYTHONPATH_EXPORT, SKYRL_DUMP_INFRA_LOG_TO_STDOUT) share a new _env_flag helper. GSM8K scoring tests gain comments explaining format vs correctness expectations.

Reviewed by Cursor Bugbot for commit 3426c47. Bugbot is set up for automated code reviews on this repo. Configure here.

Raghavan1988 and others added 7 commits August 29, 2026 22:30
`Env.init` declared `*kwargs`, which binds arbitrary positional
arguments rather than keyword arguments. Since callers and subclasses
pass configuration by keyword, the base signature should be `**kwargs`.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa
The `bool` and `int/float` branches both coerced to `float`, and `bool`
is already a subclass of `int`, so the two branches were redundant.
Collapse into a single `isinstance` check and use the imported `List`
type for consistency with the module's other annotations.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa
`hasattr(...) and getattr(...) is not None` did two attribute lookups to
express a single "present and non-null" test. `getattr` with a `None`
default collapses both into one lookup with identical behavior.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa
Three boolean environment variables each repeated the same
`str(os.environ.get(...)).lower() in ("true", "1", "yes")` incantation.
Factor it into a single `_env_flag` helper so the truthy set is defined
once and each flag reads as a one-liner.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa
`load_env_creator` unpacked `name.split(":")` directly, so a string
without exactly one colon failed with an opaque "not enough values to
unpack" ValueError. Validate the format up front and raise a message
that names the offending value and shows the expected form.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa
`_import_object` unpacked `path.rsplit(".", 1)` directly, so a path with
no dot raised an opaque unpack ValueError. Check for a dot first and
raise a message that names the bad value and the expected format.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_012Vj1rThtWbAUSYd4vuuWLa

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces several robustness improvements, bug fixes, and refactorings across the codebase, including validating import paths and entry points, fixing the signature of the init method in skyrl_gym/core.py from *kwargs to **kwargs, simplifying metric aggregation, and introducing a helper to parse boolean environment variables. The review feedback suggests hardening the new validation logic in auto.py and registration.py to handle edge cases where the input string starts or ends with the delimiter, and refactoring the _env_flag helper to avoid unnecessary string conversions of the default value.

Comment on lines +11 to +14
if "." not in path:
raise ValueError(
f"Invalid import path '{path}'. Expected a dotted 'module.path.ClassName', e.g. 'skyrl_agent.agents.runner.AgentRunner'."
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The validation only checks if . is in path. However, if path starts or ends with a dot (e.g., ".ClassName" or "module.path."), rsplit(".", 1) will produce empty strings for module_path or class_name. This can lead to unhandled ValueError (from import_module("")) or AttributeError (from getattr(..., "")).

We can make this more robust by also checking that neither split part is empty.

Suggested change
if "." not in path:
raise ValueError(
f"Invalid import path '{path}'. Expected a dotted 'module.path.ClassName', e.g. 'skyrl_agent.agents.runner.AgentRunner'."
)
if "." not in path or not all(path.rsplit(".", 1)):
raise ValueError(
f"Invalid import path '{path}'. Expected a dotted 'module.path.ClassName', e.g. 'skyrl_agent.agents.runner.AgentRunner'."
)

Comment on lines +167 to +170
if name.count(":") != 1:
raise error.Error(
f"Malformed entry point '{name}'. Expected the form '(import path):(environment name)', e.g. 'my_pkg.envs:MyEnv'."
)

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The validation checks if there is exactly one : in name. However, if name starts or ends with : (e.g., ":MyEnv" or "my_pkg:"), name.split(":") will produce empty strings for mod_name or attr_name. This can lead to unhandled ValueError (from import_module("")) or AttributeError (from getattr(..., "")).

We can make this more robust by also checking that neither split part is empty.

Suggested change
if name.count(":") != 1:
raise error.Error(
f"Malformed entry point '{name}'. Expected the form '(import path):(environment name)', e.g. 'my_pkg.envs:MyEnv'."
)
if name.count(":") != 1 or not all(name.split(":")):
raise error.Error(
f"Malformed entry point '{name}'. Expected the form '(import path):(environment name)', e.g. 'my_pkg.envs:MyEnv'."
)

Comment thread skyrl/env_vars.py

def _env_flag(name: str, default: bool = False) -> bool:
"""Parse a boolean environment variable, treating "true"/"1"/"yes" as True."""
return os.environ.get(name, str(default)).strip().lower() in _TRUTHY

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

Converting the boolean default to a string ("True" or "False") and then parsing it back via _TRUTHY is a bit indirect and less readable. It is cleaner and more idiomatic to check if the environment variable is present first, and if not, directly return the boolean default.

Suggested change
return os.environ.get(name, str(default)).strip().lower() in _TRUTHY
val = os.environ.get(name)
if val is None:
return default
return val.strip().lower() in _TRUTHY

@greptile-apps

greptile-apps Bot commented Sep 12, 2026

Copy link
Copy Markdown

RetriggerConfidence Score: 4/5

The PR appears safe to merge after aligning the base environment initialization signature with the positional prompt contract used by its implementations.

Findings

  1. P2 Base init contract mismatch

Reviews (1) · Last reviewed commit: "feat(agent): validate dotted import path..."

raise NotImplementedError

def init(self, *kwargs) -> Tuple[ObsType, Dict[str, Any]]:
def init(self, **kwargs) -> Tuple[ObsType, Dict[str, Any]]:

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Base init contract mismatch

The base Env.init method now accepts only keyword arguments, but concrete environments define init(self, prompt), callers pass prompt positionally, and the environment-authoring guide documents that positional form. This gives extension authors and static-analysis tools an inaccurate public contract; the base signature should declare the actual prompt parameter.

Suggested change
def init(self, **kwargs) -> Tuple[ObsType, Dict[str, Any]]:
def init(self, prompt: ObsType) -> Tuple[ObsType, Dict[str, Any]]:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant