feat(pd): add node-authoritative Decode admission - #1529
Open
sufubao wants to merge 10 commits into
Open
Conversation
sufubao
force-pushed
the
feat/pd-cache-aware-admission
branch
from
August 31, 2026 12:11
93312ef to
b915415
Compare
Collaborator
Author
|
极压回退修正已推送: 这次修正不再把 Radix cache 余量当作并发安全边界:
验证:104 项相关测试、200×500 随机状态转换以及 D=64 控制器饱和实验通过;GitHub pre-commit CI 已通过。控制器合成实验中,早期 cache-full 实现为 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
main中的 PD Master 在进行中请求数达到 Decode 容量时直接返回繁忙。此前版本把 admission 扩展成了 Master 侧的静态容量切分和业务调度器,但这会带来两个问题:本版本改为:Decode 节点维护唯一、权威的本地容量池;Master 只负责在多 choice 请求完成 Prefill 后协调一次原子
n槽预留。实现
Decode 节点侧 admission
running_max_req_size创建容量池,所有 Master 竞争同一个池。n个槽一次获取,不能部分准入。Prefill / Decode 两阶段
n = 1请求在 Decode 节点完成本地校验后、分配 shm request 前申请 lease。n > 1仍展开为独立的n = 1choice,以保持现有每个 choice 一条 KV 转移链路。n个槽;节点确认后才发送 Decode 请求。n > 1不做max_new_tokens分段,避免不同 continuation history 被错误拼接;n = 1保留原有分段行为。多 Master 与滚动升级
capacity_share/capacity_epoch静态切分和 Master 侧容量汇总。错误与配置
ServerBusyError语义。--pd_node_decode_admission_queue_size,默认等于running_max_req_size。--pd_node_decode_admission_timeout,默认 5 秒。--disable_pd_node_decode_admission。--disable_pd_master_decode_capacity_limit保留为 deprecated alias。精简结果
验证
变更文件通过 Black 120、Flake8 和
git diff --check。结果:
310 passed, 1 failed。唯一失败为已有的:该测试在
RadixCache.flush_cache()中要求未初始化的mem_manager非空;对应实现和测试相对upstream/main均无改动。本次未运行真实模型/GPU benchmark。