Curriculum Terms

Curriculum terms adapt the task while it trains: they widen command ranges, ramp reward weights and tighten tolerances, usually keyed off average episode length or success. The curriculum manager computes them only in training; at eval the curricula are frozen. The curricula guide covers sharing state between terms; the term catalog lists every parameter.

from eden import curricula

The base classes CurriculumTerm and CurriculumTermFuncWrapper are on the managers page.

Episode-length tracking

Locomotion curricula (e.g. average-episode-length tracking).

Attributes

BASE_NUM_ENVS module-attribute

BASE_NUM_ENVS = 4096

Classes

AverageEpisodeLengthTracker

AverageEpisodeLengthTracker(env: RLEnvBase, options: CurriculumTermOptions)

Bases: CurriculumTerm

Track the running mean of episode length.

Maintains a single device-side scalar updated with an EMA whose effective time-constant is num_compute_average_epl episodes' worth of resets at the reference env count BASE_NUM_ENVS, scaled linearly with env.num_envs.

Other curriculum terms read the current estimate via env.curriculum_manager.get_state(<this term's name>). The state dict carries both average_episode_length and an is_primed flag — the latter is False until at least one reset() call has actually fed episode lengths into the EMA, so consumers can avoid acting on the initial-zero estimate before any episode has finished.

Declare this term before any consumer in curriculum_options so the consumer's compute() sees a fresh value on the same step. Declared after, get_state raises and names the ordering rather than reading as "no value yet".

PARAMETER	DESCRIPTION
num_compute_average_epl	Reference EMA window in number of resets, expressed at BASE_NUM_ENVS. The effective window scales as max(1, int(num_compute_average_epl * num_envs / BASE_NUM_ENVS)).

Source code in eden/managers/terms/curricula/locomotion.py

def __init__(self, env: RLEnvBase, options: CurriculumTermOptions):
    super().__init__(env=env, options=options)
    self._scaled_window = max(1, int(self.num_compute_average_epl * self._env.num_envs / BASE_NUM_ENVS))
    self.average_episode_length = torch.tensor(0.0, device=self.device, dtype=torch.float)
    self._is_primed = False

Attributes

num_compute_average_epl class-attribute instance-attribute

num_compute_average_epl: int = 1000

average_episode_length instance-attribute

average_episode_length = torch.tensor(0.0, device=self.device, dtype=torch.float)

Methods:

reset

reset(envs_idx: Tensor | None = None) -> None

Source code in eden/managers/terms/curricula/locomotion.py

def reset(self, envs_idx: torch.Tensor | None = None) -> None:
    # ``envs_idx is None`` is treated as "no episodes to record" rather than
    # "all envs". CurriculumManager always passes the indices of envs that
    # just terminated; a None here would mean a startup-style call where
    # ``episode_length_buf`` is all-zero (or mid-episode), and feeding that
    # into the EMA would corrupt the running mean. Same semantics as
    # PenaltyCurriculum.reset().
    if envs_idx is None:
        return
    ep_lens = self._env.episode_length_buf[envs_idx].float()
    n = ep_lens.numel()
    if n == 0:
        return
    weight = min(n / self._scaled_window, 1.0)
    current_average = ep_lens.mean()
    # Device-side EMA update. ``lerp_`` is in-place: it avoids per-reset
    # tensor allocation on the running estimate.
    self.average_episode_length.lerp_(current_average, weight)
    self._is_primed = True

compute

compute(*args, **kwargs) -> dict[str, float | bool]

Source code in eden/managers/terms/curricula/locomotion.py

def compute(self, *args, **kwargs) -> dict[str, float | bool]:
    return {
        "average_episode_length": float(self.average_episode_length),
        "is_primed": self._is_primed,
    }

state_dict

state_dict() -> dict

Source code in eden/managers/terms/curricula/locomotion.py

def state_dict(self) -> dict:
    return {
        "average_episode_length": self.average_episode_length.detach().cpu().clone(),
        "is_primed": self._is_primed,
    }

load_state_dict

load_state_dict(state: dict) -> None

Source code in eden/managers/terms/curricula/locomotion.py

def load_state_dict(self, state: dict) -> None:
    avg = state.get("average_episode_length")
    if avg is not None:
        self.average_episode_length = torch.as_tensor(float(avg), device=self.device, dtype=torch.float)
    if "is_primed" in state:
        self._is_primed = bool(state["is_primed"])

Reward weights

Reward-weight curricula (staged ramp-up, penalty scheduling).

Attributes

Classes

StageRewardWeightCurriculum

StageRewardWeightCurriculum(env: RLEnvBase, options: CurriculumTermOptions)

Bases: CurriculumTerm

Curriculum that sets a reward term's weight at given training steps.

PARAMETER	DESCRIPTION
reward_term_name	The name of the reward term.
weight_stages	(step, weight) stages, in any order: the latest stage whose step env.common_step_counter has reached sets the weight.

Source code in eden/managers/terms/curricula/rewards.py

def __init__(self, env: RLEnvBase, options: CurriculumTermOptions):
    super().__init__(env=env, options=options)
    # make sure the stages are sorted in descending order (copy; don't mutate config)
    self.weight_stages = sorted(self.weight_stages, key=lambda x: x[0], reverse=True)

Attributes

reward_term_name class-attribute instance-attribute

reward_term_name: str = ''

weight_stages class-attribute instance-attribute

weight_stages: list[tuple[int, float]] = sorted(self.weight_stages, key=lambda x: x[0], reverse=True)

Methods:

build

build() -> None

Source code in eden/managers/terms/curricula/rewards.py

def build(self) -> None:
    # The reward manager is built before the curricula (``EnvBase._manager_order``), so an
    # unknown reward_term_name raises at env construction instead of on the first compute().
    self._reward_term = self._env.reward_manager.get_term(self.reward_term_name)

compute

compute(*args, **kwargs) -> dict[str, float]

Source code in eden/managers/terms/curricula/rewards.py

def compute(self, *args, **kwargs) -> dict[str, float]:
    # NOTE: the stages are sorted in descending order, so we can break the loop
    for stage_step, stage_weight in self.weight_stages:
        if self._env.common_step_counter >= stage_step:
            self._reward_term.weight = stage_weight
            break
    # Publish the weight actually in force -- the reward term's configured weight until the
    # first stage's step is reached, the applied stage after. Returning nothing would read back
    # through ``CurriculumManager.get_state`` as ``None`` (indistinguishable from "never computed")
    # and log no ``Curriculum/`` key at all.
    return {"weight": float(self._reward_term.weight)}

PenaltyCurriculum

PenaltyCurriculum(env: RLEnvBase, options: CurriculumTermOptions)

Bases: CurriculumTerm

Curriculum that gradually scales reward weights for a set of reward terms.

Monitors average episode length and adjusts a scale factor applied to all targeted reward terms. When episodes are short (agent struggling), the scale decreases to ease penalties. When episodes are long (agent succeeding), the scale increases to enforce stricter behaviour.

The scale factor multiplies the configured weight of each term, so the effective weight is configured_weight * scale.

Targets can be specified by name OR by tag — exactly one entry path must be used per curriculum instance. Mixing the two would split intent across two fields and make refactors error-prone (e.g. renaming a term forgets the name-list, but the tag-list still matches). Prefer reward_term_tags when the same curriculum should apply to a group of penalties whose roster may grow (just tag new terms); use reward_term_names for explicit, audited targets.

PARAMETER	DESCRIPTION
reward_term_names	Explicit names of reward terms to scale. Mutually exclusive with reward_term_tags.
reward_term_tags	Tags to match on RewardTermOptions.tags. A term is selected if any of its tags appears in this list. Mutually exclusive with reward_term_names.
initial_scale	Starting scale factor. Default 0.1 (10% of configured weight).
min_scale	Minimum allowed scale. Default 0.0.
max_scale	Maximum allowed scale. Default 1.0.
level_up_threshold	Average episode length (in steps) above which the scale increases.
level_down_threshold	Average episode length (in steps) below which the scale decreases.
degree	Amount to adjust the scale per curriculum compute call.
num_episode_average	Number of most recent episode lengths to average over. Ignored when tracker_name is set (the named tracker owns the EMA).
tracker_name	Optional name of an AverageEpisodeLengthTracker curriculum term whose average_episode_length should drive this curriculum instead of the per-instance buffer. The tracker must be declared before this PenaltyCurriculum in curriculum_options so the tracker's compute() runs first within a step. When None (default), a per-instance CPU buffer of the last num_episode_average lengths is used.

Source code in eden/managers/terms/curricula/rewards.py

def __init__(self, env: RLEnvBase, options: CurriculumTermOptions):
    super().__init__(env=env, options=options)
    if not self.reward_term_names and not self.reward_term_tags:
        raise ValueError("PenaltyCurriculum requires either `reward_term_names` or `reward_term_tags` to be set.")
    if self.reward_term_names and self.reward_term_tags:
        raise ValueError(
            "PenaltyCurriculum: `reward_term_names` and `reward_term_tags` are mutually exclusive — "
            "set exactly one. Use names for explicit targets, tags for groups whose roster may grow."
        )
    self._scale = self.initial_scale
    self._targeted_terms: list = []
    self._base_weights: list[float] = []
    if self.tracker_name is None:
        # Rolling buffer for episode lengths. Kept on CPU on purpose: the running
        # mean is consumed by compute() every env step and used to branch on
        # ``level_up_threshold`` / ``level_down_threshold``. A device-side mean
        # would force a GPU→CPU sync per step, defeating async kernel launch.
        # Episode lengths are tiny ints; the CPU buffer is cheap.
        self._episode_lengths = torch.zeros(self.num_episode_average, device="cpu")
        self._ep_cursor = 0
        self._ep_count = 0

Attributes

reward_term_names class-attribute instance-attribute

reward_term_names: list[str] = []

reward_term_tags class-attribute instance-attribute

reward_term_tags: list[str] = []

initial_scale class-attribute instance-attribute

initial_scale: float = 0.1

min_scale class-attribute instance-attribute

min_scale: float = 0.0

max_scale class-attribute instance-attribute

max_scale: float = 1.0

level_up_threshold class-attribute instance-attribute

level_up_threshold: float = 750.0

level_down_threshold class-attribute instance-attribute

level_down_threshold: float = 150.0

degree class-attribute instance-attribute

degree: float = 0.00025

num_episode_average class-attribute instance-attribute

num_episode_average: int = 1000

tracker_name class-attribute instance-attribute

tracker_name: str | None = None

Methods:

build

build() -> None

Resolve the targeted reward terms and install initial_scale on them.

The reward manager is built before the curricula (EnvBase._manager_order), so an unknown reward_term_names entry raises at env construction rather than on the first reset. Installing the scale here — rather than on whichever of reset() / compute() runs first — is also what keeps the initial scale applied at eval: CurriculumManager freezes both of those there, so a term that only ever installed its starting level from one of them would grade an eval rollout at the unscaled configured weights.

Source code in eden/managers/terms/curricula/rewards.py

def build(self) -> None:
    """Resolve the targeted reward terms and install ``initial_scale`` on them.

    The reward manager is built before the curricula (``EnvBase._manager_order``), so an
    unknown ``reward_term_names`` entry raises at env construction rather than on the first
    reset. Installing the scale here — rather than on whichever of ``reset()`` / ``compute()``
    runs first — is also what keeps the initial scale applied at eval:
    ``CurriculumManager`` freezes both of those there, so a term that only ever installed its
    starting level from one of them would grade an eval rollout at the *unscaled* configured
    weights.
    """
    reward_manager = self._env.reward_manager
    if self.reward_term_names:
        # Explicit names: error if any is missing.
        self._targeted_terms = [reward_manager.get_term(name) for name in self.reward_term_names]
    else:
        # Tag match: scan all terms and keep those carrying any listed tag.
        tag_set = set(self.reward_term_tags)
        self._targeted_terms = [
            term
            for _, term in reward_manager.iter_terms()
            if tag_set.intersection(getattr(term, "tags", None) or [])
        ]
    self._base_weights = [term.weight for term in self._targeted_terms]
    # Empty resolution is allowed (e.g. tag-based curriculum on a task that doesn't
    # use that tag yet) but warn loudly so it isn't a silent no-op.
    if not self._targeted_terms:
        en.logger.warning(
            "PenaltyCurriculum resolved zero target reward terms "
            f"(reward_term_names={self.reward_term_names!r}, reward_term_tags={self.reward_term_tags!r}). "
            "The curriculum will be a no-op — check that names exist and that target terms carry the listed tags."
        )
    self._apply_scale()

reset

reset(envs_idx: Tensor | None = None) -> None

Source code in eden/managers/terms/curricula/rewards.py

def reset(self, envs_idx: torch.Tensor | None = None) -> None:
    if self.tracker_name is not None:
        # The named tracker owns the EMA; nothing to update here.
        return
    if envs_idx is None:
        return
    # Pull episode lengths to CPU once per reset and update the running mean
    # incrementally (avoids per-step ``.mean().item()`` syncs in compute()).
    ep_lens = self._env.episode_length_buf[envs_idx].float().cpu()
    n = ep_lens.shape[0]
    if n == 0:
        return
    buf_size = self.num_episode_average
    indices = (torch.arange(n, device=self._episode_lengths.device) + self._ep_cursor) % buf_size
    self._episode_lengths[indices] = ep_lens
    self._ep_cursor = (self._ep_cursor + n) % buf_size
    self._ep_count = min(self._ep_count + n, buf_size)
    # Refresh cached running mean as a Python float — no per-step sync below.
    self._avg_ep_len = float(self._episode_lengths[: self._ep_count].mean())

compute

compute(*args, **kwargs) -> dict[str, float]

Source code in eden/managers/terms/curricula/rewards.py

def compute(self, *args, **kwargs) -> dict[str, float]:
    if self.tracker_name is not None:
        # A tracker that is registered but hasn't computed yet raises inside get_state: it must
        # be declared *before* this term in curriculum_options, and the manager says so.
        avg_ep_len, is_primed = self._read_tracker_state()
        # Same gate as the buffer path's ``_ep_count > 0``: only move the scale once
        # the tracker has actually consumed at least one reset. Otherwise the
        # initial-zero estimate would push ``_scale`` toward ``min_scale`` on
        # every pre-first-reset step.
        if is_primed:
            if avg_ep_len > self.level_up_threshold:
                self._scale = min(self._scale + self.degree, self.max_scale)
            elif avg_ep_len < self.level_down_threshold:
                self._scale = max(self._scale - self.degree, self.min_scale)
            self._apply_scale()
        return {"scale": self._scale, "avg_episode_length": avg_ep_len}

    avg_ep_len = getattr(self, "_avg_ep_len", 0.0)

    # Only update the scale once we have at least one recorded episode length.
    # Otherwise compute() runs every env.step() from start of training and the
    # default avg_ep_len=0.0 would push _scale toward min_scale before any
    # real episode data exists.
    if self._ep_count > 0:
        if avg_ep_len > self.level_up_threshold:
            self._scale = min(self._scale + self.degree, self.max_scale)
        elif avg_ep_len < self.level_down_threshold:
            self._scale = max(self._scale - self.degree, self.min_scale)
        self._apply_scale()
    return {"scale": self._scale, "avg_episode_length": avg_ep_len}

Velocity commands

Velocity curriculum for progressive velocity range expansion.

Attributes

Classes

VelocityCommandCurriculum

VelocityCommandCurriculum(env: RLEnvBase, options: CurriculumTermOptions)

Bases: CurriculumTerm

Curriculum that progressively expands velocity command ranges.

PARAMETER	DESCRIPTION
command_name	The name of the velocity command term in the command manager. TYPE: str
velocity_stages	Stages, in any order (sorted by step on construction). Each dict has step (int) and optional lin_vel_x, lin_vel_y, ang_vel_z tuples for the new ranges. Example:: [ {"step": 0, "lin_vel_x": (-0.5, 1.0), "lin_vel_y": (-0.5, 0.5)}, {"step": 120000, "lin_vel_x": (-1.0, 2.0), "lin_vel_y": (-1.0, 1.0)}, ] TYPE: list[dict]

Source code in eden/managers/terms/curricula/velocity.py

def __init__(self, env: RLEnvBase, options: CurriculumTermOptions):
    super().__init__(env=env, options=options)
    self.velocity_stages = sorted(self.velocity_stages, key=lambda x: x["step"], reverse=True)
    self.command_term = self._env.command_manager.get_term(self.command_name)
    self._current_stage_step = -1

Attributes

command_name class-attribute instance-attribute

command_name: str = 'velocity_command'

velocity_stages class-attribute instance-attribute

velocity_stages: list[dict] = sorted(self.velocity_stages, key=lambda x: x['step'], reverse=True)

command_term instance-attribute

command_term = self._env.command_manager.get_term(self.command_name)

Methods:

compute

compute(*args, **kwargs)

Source code in eden/managers/terms/curricula/velocity.py

def compute(self, *args, **kwargs):
    for stage in self.velocity_stages:
        if self._env.common_step_counter >= stage["step"]:
            if stage["step"] != self._current_stage_step:
                self._current_stage_step = stage["step"]
                if "lin_vel_x" in stage:
                    self.command_term.lin_vel_x_range = tuple(stage["lin_vel_x"])
                if "lin_vel_y" in stage:
                    self.command_term.lin_vel_y_range = tuple(stage["lin_vel_y"])
                if "ang_vel_z" in stage:
                    self.command_term.ang_vel_z_range = tuple(stage["ang_vel_z"])
            break
    return {"stage_step": self._current_stage_step}

Motion tracking

Motion-tracking curricula (success-driven tolerance tightening).

Attributes

Classes

MotionBasePoseThresholdCurriculum

MotionBasePoseThresholdCurriculum(env: EnvBase, options: _TermOptionsT)

Bases: CurriculumTerm

Tighten a :class:MotionBasePoseTrackingBreak gate as success rises.

ConTrack's "threshold curriculum": an EMA of the per-episode success rate linearly interpolates the paired termination term's pos_threshold / rot_threshold from their *_max (lenient) values to their *_min (tight) ones as the EMA moves across [success_rate_low, success_rate_high]. An episode counts as a success when it ends with the motion clip exhausted (validity[:, 0] false at reset time) rather than via the pose gate.

This lives in a curriculum term rather than inside the termination term so it gets the three things curriculum state needs and a termination term cannot provide: CurriculumManager logs :meth:compute's return under Curriculum/<name>/*, it freezes curricula at eval (eval_mode=True) so an eval rollout is graded at a fixed tolerance, and the state has one obvious home if/when term checkpointing lands (nothing calls state_dict() on manager terms today, so a resumed run still restarts the EMA at 0 — i.e. at the lenient thresholds).

Declaration order matters only against other curriculum terms; the thresholds this writes are consumed by the next step's termination_manager.compute().

PARAMETER	DESCRIPTION
termination_term_name	Name of the MotionBasePoseTrackingBreak term to drive, as declared in termination_options.
entity_name	Entity whose motion-reference validity defines episode success. Must match the termination term's own entity_name.
pos_threshold_max	Position tolerance (m) at success_rate_low / success_rate_high.
rot_threshold_max	Orientation tolerance (rad) at success_rate_low / success_rate_high.
success_ema_alpha	Per-episode EMA rate. The update is applied once per reset batch with an effective rate of 1 - (1 - alpha) ** n for a batch of n episodes, which is exactly n successive per-episode updates. Without that correction the EMA's time constant would scale with num_envs (at 4096 envs some env resets almost every step, so a per-batch alpha converges within a fraction of one episode).
success_rate_low	EMA values at which the thresholds sit at *_max and *_min respectively.

Source code in eden/managers/base.py

def __init__(self, env: EnvBase, options: _TermOptionsT):
    ManagerTermBase.__init__(self, env=env)
    ConfigurableMixin.__init__(self, options=options)

Attributes

termination_term_name class-attribute instance-attribute

termination_term_name: str = 'object_pose_break'

entity_name class-attribute instance-attribute

entity_name: str = 'object'

pos_threshold_max class-attribute instance-attribute

pos_threshold_max: float = 0.1

pos_threshold_min class-attribute instance-attribute

pos_threshold_min: float = 0.05

rot_threshold_max class-attribute instance-attribute

rot_threshold_max: float = 1.0

rot_threshold_min class-attribute instance-attribute

rot_threshold_min: float = 0.5

success_ema_alpha class-attribute instance-attribute

success_ema_alpha: float = 0.05

success_rate_low class-attribute instance-attribute

success_rate_low: float = 0.9

success_rate_high class-attribute instance-attribute

success_rate_high: float = 0.999

Methods:

build

build() -> None

Source code in eden/managers/terms/curricula/motion.py

def build(self) -> None:
    # Fail fast on the degenerate spans: `high == low` would make the interpolation
    # weight 0/0 -> nan, which propagates through clamp into both thresholds and turns
    # `err > threshold` permanently False — the gate would silently never fire.
    if not self.success_rate_high > self.success_rate_low:
        raise ValueError(
            f"{term_subject(self)} requires success_rate_high > success_rate_low; got "
            f"success_rate_low={self.success_rate_low}, success_rate_high={self.success_rate_high}."
        )
    if not 0.0 < self.success_ema_alpha <= 1.0:
        raise ValueError(f"{term_subject(self)} requires 0 < success_ema_alpha <= 1; got {self.success_ema_alpha}.")
    for name in ("pos", "rot"):
        lo, hi = getattr(self, f"{name}_threshold_min"), getattr(self, f"{name}_threshold_max")
        if lo > hi:
            raise ValueError(
                f"{term_subject(self)}: {name}_threshold_min ({lo}) must not exceed "
                f"{name}_threshold_max ({hi}) — the curriculum tightens from max to min."
            )
    # The reset grades the episode that ended, whose window is refilled before the manager resets
    # run; a wrong entity_name raises here, not on the first reset.
    self._env.motion_manager.record_ended_episode_validity(self.entity_name)
    self._success_ema = torch.zeros((), device=self.device)
    # The termination manager is built before the curricula (``EnvBase._manager_order``), so
    # the term this curriculum drives resolves here — a wrong name or a mismatched entity
    # raises at env construction rather than on the first compute().
    self._term = self._env.termination_manager.get_term(self.termination_term_name)
    if not isinstance(self._term, MotionBasePoseTrackingBreak):
        raise TypeError(
            f"{term_subject(self)} drives termination term '{self.termination_term_name}', a "
            f"{type(self._term).__name__}, but it can only move a MotionBasePoseTrackingBreak gate (through its "
            "set_thresholds). Point termination_term_name at one."
        )
    if self._term.entity_name != self.entity_name:
        raise ValueError(
            f"{term_subject(self)} drives termination term '{self.termination_term_name}' on entity "
            f"'{self._term.entity_name}' but grades success on '{self.entity_name}'. Point both at "
            f"the same entity, or the tolerance would track a different clip's progress."
        )

reset

reset(envs_idx: slice | Tensor | None = None) -> None

Source code in eden/managers/terms/curricula/motion.py

def reset(self, envs_idx: slice | torch.Tensor | None = None) -> None:
    env: MimicRLEnvBase = self._env
    # Both reads below describe the episode that just ended: the validity the motion manager took
    # as it ended (the window itself already holds the new episode's), and `episode_length_buf`,
    # which `_reset_idx` zeroes after the manager resets. An exhausted clip (the episode ran to
    # the end instead of breaking) reads as validity False.
    success = (~env.motion_manager.ended_episode_validity(self.entity_name)).to(torch.float32)
    # Never-stepped envs (the build-time bootstrap reset) still hold the all-invalid
    # make_empty default, which would read as instant success and bias the EMA.
    stepped = (env.episode_length_buf > 0).to(torch.float32)
    # Kept entirely device-side: no `.item()`/`bool()`/mask-index — the auto-reset hands a bool
    # mask, which as an index is a sync per gather, so the sums reduce through it instead (they
    # count whole numbers, so they match the gathered ones exactly). n == 0 makes `weight` 0,
    # i.e. a no-op update.
    envs_idx = slice(None) if envs_idx is None else envs_idx
    n = sum_over_envs(stepped, envs_idx)
    mean_success = sum_over_envs(success * stepped, envs_idx) / n.clamp(min=1.0)
    weight = 1.0 - (1.0 - self.success_ema_alpha) ** n
    self._success_ema += weight * (mean_success - self._success_ema)

compute

compute(*args, **kwargs) -> dict[str, float]

Source code in eden/managers/terms/curricula/motion.py

def compute(self, *args, **kwargs) -> dict[str, float]:
    w = torch.clamp(
        (self._success_ema - self.success_rate_low) / (self.success_rate_high - self.success_rate_low),
        0.0,
        1.0,
    )
    pos_threshold = self.pos_threshold_max * (1.0 - w) + self.pos_threshold_min * w
    rot_threshold = self.rot_threshold_max * (1.0 - w) + self.rot_threshold_min * w
    self._term.set_thresholds(pos_threshold, rot_threshold)
    return {
        "success_ema": self._success_ema,
        "pos_threshold": pos_threshold,
        "rot_threshold": rot_threshold,
    }

state_dict

state_dict() -> dict

Source code in eden/managers/terms/curricula/motion.py

def state_dict(self) -> dict:
    return {"success_ema": self._success_ema.detach().cpu().clone()}

load_state_dict

load_state_dict(state: dict) -> None

Source code in eden/managers/terms/curricula/motion.py

def load_state_dict(self, state: dict) -> None:
    if (ema := state.get("success_ema")) is not None:
        self._success_ema = torch.as_tensor(float(ema), device=self.device, dtype=torch.float)

Functions:

Success Guided Sampling

Success Guided Sampling (SGS): choose each episode's reset configuration from its recent success rate.

Zhang et al., "A Balanced Data Diet: Addressing the Exploration Bottleneck in Mega-Scale RL for Robot Control" (CoRL 2026, arXiv:2610.12465), Sec. 3.2 and Appendix B.2. A task owns a fixed set of N reset configurations. SGS keeps the last history success/failure outcomes of every configuration, scores each configuration's success rate p with a Beta-shaped kernel peaked at target, and draws the next configuration of every resetting env from a softmax over those scores. Most episodes go to configurations the policy solves sometimes; every configuration keeps a non-zero probability, so the estimates of the ones it cannot solve yet (or has mastered) keep being refreshed.

The term is a :class:~eden.managers.resets.sampling.ResetConfigSampler: the RESET event that owns the configuration set binds it and calls :meth:SuccessGuidedSampling.sample, because the RESET events run before any manager's reset() (see :mod:eden.managers.resets.sampling). sample grades the episode that just ended before it draws the next one -- the termination manager still holds the finished step's verdicts then, since its own reset() comes later in the same walk.

Attributes

Classes

SuccessGuidedSampling

SuccessGuidedSampling(env: EnvBase, options: _TermOptionsT)

Bases: CurriculumTerm, ResetConfigSampler

Draw each resetting env's reset configuration with probability peaked at a target success rate.

A reset event that owns a configuration set names this term (its sampler parameter), binds it to the set size in its build() and calls :meth:sample from every reset. A term no reset event binds fails env construction. A configuration never graded reads p = 0, so before any outcome is in, sampling is uniform.

An outcome is recorded for an env when its episode ended: it is in the reset selection, the termination manager has it done, and it ran at least one step. A success is the success_term termination having fired on that last step. Not graded: an env's first spawn, an env.reset() that cut an episode short, and an episode the simulator flagged as corrupt (termination_manager.solver_errors) -- that one ended on the solver, not on the task. When more than history envs finish one configuration in the same reset, a uniformly random history of them are recorded.

At eval (env.training False) configurations are drawn uniformly and nothing is recorded: evaluation grades the policy on the task distribution, not on the sampler's current emphasis.

The defaults are the paper's locomotion settings; it uses target=0.5, concentration=1.0, floor=1e-4 for manipulation.

ATTRIBUTE	DESCRIPTION
success_term	Termination term whose firing marks an episode a success (e.g. all_goals_reached). TYPE: str | None
history	Outcomes kept per configuration (a ring buffer); p is their mean. TYPE: int
target	Success rate the kernel peaks at, in [0, 1]. TYPE: float
concentration	Kernel sharpness k >= 0; 0 samples uniformly. TYPE: float
temperature	Softmax temperature, effective value max(temperature, 1). TYPE: float
floor	eps > 0: keeps the kernel finite at p in {0, 1} and every probability non-zero. TYPE: float

Source code in eden/managers/base.py

def __init__(self, env: EnvBase, options: _TermOptionsT):
    ManagerTermBase.__init__(self, env=env)
    ConfigurableMixin.__init__(self, options=options)

Attributes

success_term class-attribute instance-attribute

success_term: str | None = None

history class-attribute instance-attribute

history: int = 100

target class-attribute instance-attribute

target: float = 0.66

concentration class-attribute instance-attribute

concentration: float = 5.0

temperature class-attribute instance-attribute

temperature: float = 2.0

floor class-attribute instance-attribute

floor: float = 1e-08

success_rate property

success_rate: Tensor

(num_configs,): mean of each configuration's recorded outcomes; 0 where none is recorded.

num_outcomes property

num_outcomes: Tensor

(num_configs,) long: outcomes recorded per configuration over the run (not capped at history).

probabilities property

probabilities: Tensor

(num_configs,): the distribution the next training reset draws from.

Methods:

build

build() -> None

Source code in eden/managers/terms/curricula/success_guided.py

def build(self) -> None:
    self._validate_params()
    terminations = getattr(self._env, "termination_manager", None)
    if terminations is None:
        raise ValueError(
            "SuccessGuidedSampling grades episodes by a termination term, but this env has no termination "
            "manager. Fix: build it from an EdenRLConfig with termination_options."
        )
    if self.success_term is None or self.success_term not in terminations.active_terms:
        raise ValueError(
            f"SuccessGuidedSampling: success_term={self.success_term!r} is not an active termination term. "
            f"Active termination terms: {', '.join(terminations.active_terms) or '(none)'}.\n"
            f"Fix: SuccessGuidedSampling.configure(success_term='<one of those>') -- the term whose firing "
            "means the episode succeeded."
        )
    with torch.inference_mode(False):
        self._env_arange = torch.arange(self.num_envs, device=self.device)
    self._require_bound_at_build(self._env)

compute

compute() -> dict[str, Tensor]

Publish the sampler's state; device tensors, so logging it costs no sync.

mean_success averages p over the configurations graded at least once; frontier_frac is the share of the whole set with 0 < p < 1 -- solved sometimes, the configurations SGS exists to find; sampling_ess_frac is the effective sample size of the sampling distribution as a fraction of the set (1 = uniform).

Source code in eden/managers/terms/curricula/success_guided.py

def compute(self) -> dict[str, torch.Tensor]:
    """Publish the sampler's state; device tensors, so logging it costs no sync.

    ``mean_success`` averages ``p`` over the configurations graded at least once;
    ``frontier_frac`` is the share of the whole set with ``0 < p < 1`` -- solved sometimes, the
    configurations SGS exists to find; ``sampling_ess_frac`` is the effective sample size of the
    sampling distribution as a fraction of the set (1 = uniform).
    """
    self._require_bound("compute()")
    visited = self.num_outcomes > 0
    num_visited = visited.sum()
    p = self._success_rate
    return {
        "mean_success": (p * visited).sum() / num_visited.clamp(min=1),
        "visited_frac": num_visited / self._num_configs,
        "frontier_frac": ((p > 0.0) & (p < 1.0)).float().mean(),
        "sampling_ess_frac": 1.0 / (self._probs.square().sum() * self._num_configs),
    }

Functions:

success_guided_probabilities

success_guided_probabilities(success_rate: Tensor, *, target: float, concentration: float, temperature: float, floor: float) -> Tensor

Sampling probability of each configuration given its success rate (paper Eq. 1-2, Appendix B.2).

w = (p + eps)^(k t) * (1 - p + eps)^(k (1 - t)) -- the Beta(1 + k t, 1 + k (1 - t)) density in mode-concentration form, up to a constant -- floored at eps; the logit is log(w + eps) and the probabilities a softmax over the logits at temperature max(temperature, 1).

Source code in eden/managers/terms/curricula/success_guided.py

def success_guided_probabilities(
    success_rate: torch.Tensor, *, target: float, concentration: float, temperature: float, floor: float
) -> torch.Tensor:
    """Sampling probability of each configuration given its success rate (paper Eq. 1-2, Appendix B.2).

    ``w = (p + eps)^(k t) * (1 - p + eps)^(k (1 - t))`` -- the Beta(1 + k t, 1 + k (1 - t)) density in
    mode-concentration form, up to a constant -- floored at ``eps``; the logit is ``log(w + eps)`` and the
    probabilities a softmax over the logits at temperature ``max(temperature, 1)``.
    """
    p = success_rate.float()
    weight = (p + floor).pow(concentration * target) * (1.0 - p + floor).pow(concentration * (1.0 - target))
    logits = torch.log(weight.clamp(min=floor) + floor)
    return torch.softmax(logits / max(temperature, 1.0), dim=-1)

See also

• Curricula guide
• Curriculum manager options