docs: document PyTorch fused AdamW opt-in

[adenzhou1350]

Summary

Document how to select PyTorch's built-in fused AdamW through DeepSpeed configuration:

{
  "optimizer": {
    "type": "AdamW",
    "params": {
      "torch_adam": true,
      "fused": true
    }
  }
}

The documentation explains that this avoids building DeepSpeed's native fused Adam extension and preserves PyTorch's compatibility, memory, and floating-point caveats. It does not change any runtime default.

Motivation and validation

On a Qwen3-4B-Base, sequence-2048, BF16, ZeRO-2 DP4 workload with four RTX 5090 GPUs, the existing opt-in reduced the measured optimizer-update phase from about 220 ms to 18.5 ms. Four order-balanced blocks measured a median whole-step speedup of 1.15845x, with a bootstrap 95% interval of [1.15329, 1.16659].

Two reverse-order 20-step trajectories reproduced exactly within each optimizer arm and passed the repository's BF16 tolerance contract. These numbers motivate documenting the opt-in; they do not justify a global default or extrapolation to other optimizers, offload modes, models, runtimes, or hardware.

Applicable pre-commit hooks and git diff --check pass.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 4db136cd32

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".