记号
Machine Learning Systems spans machine learning (computer science and statistics) and systems (computer architecture and hardware). Each field developed its notation independently, and many symbols mean different things across communities and publications. This collision creates real confusion when the disciplines merge. The conventions below establish a single notation that eliminates this ambiguity.
Consider a simple statement: “Increasing \(B\) improves throughput.” To an ML researcher, \(B\) means batch size. To a hardware engineer, \(B\) means bandwidth. Both interpretations are correct in their respective fields, but in ML Systems we need both concepts in the same equation—hence the need for a single, consistent convention.
The Iron Law of ML Systems
For serialized execution phases, the iron-law performance decomposition is:
\[T = \underbrace{\frac{D_{\text{vol}}}{\text{BW}}}_{\text{Memory Time}} + \underbrace{\frac{O}{R_{\text{peak}} \cdot \eta_{\text{hw}}}}_{\text{Compute Time}} + \underbrace{L_{\text{lat}}}_{\text{Latency Overhead}}\]
Each variable was chosen deliberately to avoid collision with standard ML terminology.
| Symbol | Definition | Unit | Why This Symbol? |
|---|---|---|---|
| \(T\) | Time | seconds | Unambiguous. Wall-clock time for an operation. |
| \(D_{\text{vol}}\) | Data Volume | bytes | Avoids collision with \(D\) (Dataset Size). In scaling laws, \(D\) means training tokens. Here we need bytes moved through memory. The subscript disambiguates. |
| \(\text{BW}\) | Bandwidth | bytes/s | Avoids collision with \(B\) (Batch Size). Systems literature often uses \(B\) for bandwidth, while ML literature often uses \(B\) for batch size. The notation preserves the ML convention. |
| \(O\) | Operations | FLOPs | Total floating-point operations. Clean in equations (vs. “\(\text{Ops}\)”). |
| \(R_{\text{peak}}\) | Peak Rate | FLOP/s | Avoids collision with \(P\) (Parameters). Roofline presentations often use \(P\) for performance, while ML literature commonly uses \(P\) for parameter count. The notation preserves the ML convention. |
| \(\eta_{\text{hw}}\) | Efficiency | — | Hardware utilization \((0 \le \eta_{\text{hw}} \le 1)\). Avoids collision with learning rate \((\eta)\). |
| \(L_{\text{lat}}\) | Latency | seconds | Avoids collision with \(\mathcal{L}\) (Loss). Fixed overhead time (kernel launch, network RTT). The subscript distinguishes from the loss function. |
Why these choices matter
Without careful notation, sentences become ambiguous:
“Reducing \(D\) improves performance.”
The sentence has two possible readings:
- Reducing dataset size (fewer training samples) can speed training but may reduce accuracy.
- Reducing data volume moved (for example, through compression or quantization) can speed inference, with accuracy effects that depend on the technique and calibration.
With our notation, we can write precisely:
“Reducing \(D_{\text{vol}}\) through FP32-to-INT8 quantization cuts parameter memory traffic to one quarter while \(D\) (training data) remains unchanged.”
Our notation eliminates this ambiguity: “\(\text{BW}\) limits throughput” has only one reading.
Subscripted variants
An upright root names the quantity, and the subscript names which instance of it is meant. The convention holds closely related measurements apart instead of collapsing them into a single symbol:
- \(\text{BW}_{\text{disk}}\), \(\text{BW}_{\text{network}}\), \(\text{BW}_{\text{accelerator}}\): bandwidth at three points on the data path
- \(D_{\text{vol}}\), \(R_{\text{peak}}\), \(L_{\text{lat}}\), \(\eta_{\text{hw}}\): iron-law terms, each held clear of a common ML symbol
- \(E_{\text{move}}\), \(E_{\text{compute}}\), \(E_{\text{total}}\): one energy budget split into tradeable parts
The subscript therefore carries part of the claim. Disk and accelerator bandwidth differ by orders of magnitude, so a figure is reproducible only when the subscript says where it was measured.
The Degradation Equation
The degradation equation is a fitted local model. Divergence alone does not determine the direction of accuracy change, so estimate \(\lambda\) from labeled outcomes. Some symbols below, such as \(\tau\), appear in chapter prose rather than in the equation. \[\text{Accuracy}(t) \approx \text{Accuracy}_0 - \lambda \cdot \mathcal{D}(P_t \lVert P_0)\]
| Symbol | Definition | Unit/Type | Notes |
|---|---|---|---|
| \(\text{Accuracy}(t)\) | Accuracy at Time \(t\) | Scalar | Model accuracy after the model has been deployed for time \(t\). |
| \(\text{Accuracy}_0\) | Initial Accuracy | Scalar | Model accuracy at deployment time. |
| \(\lambda\) | Fitted Sensitivity | Scalar | Local coefficient estimated from outcomes over a stated range; not a universal constant. (Not wavelength.) |
| \(P_t\) | Current Distribution | Distribution | The data distribution at time \(t\). (Not parameters—use \(P\) for parameter count.) |
| \(P_0\) | Training Distribution | Distribution | The data distribution at training time. |
| \(\mathcal{D}(P_t \lVert P_0)\) | Statistical Divergence | Scalar \(\ge 0\) | Measures how far \(P_t\) has drifted from \(P_0\). Common choices: KL divergence, total variation, Wasserstein. (Calligraphic to avoid collision with \(D\) = dataset size.) |
| \(\tau\) | Response Threshold | Scalar \(> 0\) | Operational threshold for investigation; it does not automatically trigger retraining. |
The Energy Corollary
With effective per-byte and per-operation costs, workload energy is approximated as: \[E_{\text{total}} \approx D_{\text{vol}} \times E_{\text{move}} + O \times E_{\text{compute}}\]
| Symbol | Definition | Unit | Notes |
|---|---|---|---|
| \(E_{\text{total}}\) | Total Energy | joules | Total energy consumed by an ML workload, decomposed into data-movement and compute terms. |
| \(E_{\text{move}}\) | Energy per Byte Moved | joules/byte | Effective movement cost; total energy also depends on bytes moved and operations executed. |
| \(E_{\text{compute}}\) | Energy per Operation | joules/FLOP | Energy cost of a single arithmetic operation. |
Deep Learning Notation
This book follows standard deep learning conventions (Goodfellow 等 2016年) with explicit disambiguation for systems variables.
| Symbol | Definition | Dimensions/Type |
|---|---|---|
| \(B\) | Batch Size | Integer. The number of samples processed in parallel. (Never bandwidth.) |
| \(P\) | Parameters | Integer. The total count of scalar model parameters, including weights and biases. (Never peak FLOP/s.) |
| \(D\) | Dataset Size | Integer. Number of training samples or tokens. (Never data volume in bytes—use \(D_{\text{vol}}\).) |
| \(S\) | Sequence Length | Integer. Number of tokens or time steps. |
| \(d\) | Hidden Dimension | Integer. Size of the hidden state vector. |
| \(d_{\text{head}}\) | Attention Head Dimension | Integer. Per-head hidden dimension in attention layers. |
| \(N_L\) | Number of Layers | Integer. Total number of layers in a network. |
| \(N_{\text{heads}}\) | Number of Attention Heads | Integer. Number of attention heads in a multi-head attention layer. |
| \(H_{\text{KV}}\) | Number of Key-Value Heads | Integer. Number of key-value heads in grouped-query or multi-query attention. |
| \(\ell\) | Layer Index | Integer. Index for a layer. Use instead of bare \(L\) when indexing layers, since \(L\) collides with loss and latency. |
| \(\mathcal{L}\) | Loss Function | Scalar. The objective function minimized during training. |
| \(\eta\) | Learning Rate | Scalar. Step size for the optimizer. (Never bare for hardware efficiency—use \(\eta_{\text{hw}}\).) |
| \(\theta\) | Model Parameters | Parameter collection, often treated as a vector. The set of all learnable parameters. |
| \(p(x)\) | Distribution | Probability mass/density for a random variable. Use lowercase \(p(\cdot)\) for generic distributions to avoid collision with \(P\) (parameter count). |
| \(p(y \mid x)\) | Conditional Distribution | Conditional probability/density. Use this form for generic label relationships; reserve \(P_0\) and \(P_t\) for the degradation equation’s training/current distributions. |
| \(\Pr(E)\) | Event Probability | Probability of an event \(E\). Use for event statements such as \(\Pr(\text{batch}=0)\); use \(p(x)\) and \(p(y \mid x)\) for distributions. |
Latin matrices and vectors are set in bold (\(\mathbf{W}\), \(\mathbf{x}\)); scalars, dimensions, indices, and individual matrix entries stay italic (\(N\), \(d\), \(i\), \(W_{ij}\)). The generic parameter symbol \(\theta\) is the one conventional exception and stays italic. Local matrix algebra follows the same bold rule: matrix operands are bold (\(\mathbf{A}\), \(\mathbf{B}\), \(\mathbf{C}\)), as in general matrix multiply (GEMM) \(\mathbf{C}=\alpha \mathbf{A}\mathbf{B}+\beta \mathbf{C}\). The bold form marks a matrix operand and is distinct from the italic scalar batch size \(B\), which remains batch size in scalar contexts.
Performance, Serving, and Memory Notation
Reusable performance and serving quantities follow the same collision-avoidance rule as the iron law. Rates use \(R\) or descriptive Greek symbols; request counts use \(Q_{\text{req}}\) rather than overloading \(N\); queue utilization uses a subscripted \(\rho\) so bare \(\rho\) remains available for other ratio models.
| Symbol | Definition | Unit/Type | Notes |
|---|---|---|---|
| \(I\) | Arithmetic Intensity | FLOP/byte | Workload FLOPs per byte moved. The Roofline Model uses \(I\) as the independent variable. |
| \(I_{\text{ridge}}\) | Roofline Ridge Point | FLOP/byte | \(I_{\text{ridge}} = R_{\text{peak}}/\text{BW}\). Prefer this explicit form over starred shorthand so the meaning remains clear in prose. |
| \(R_{\text{attain}}\) | Attainable Compute Rate | FLOP/s | Roofline bound: \(R_{\text{attain}} \leq \min(R_{\text{peak}}, I \times \text{BW})\). Uses \(R\), not \(T\), because this quantity is a rate. |
| \(\text{MFU}\) | Model FLOPs Utilization | Dimensionless | Useful model FLOP/s divided by available peak FLOP/s. Text acronym avoids overloading \(\eta\). |
| \(r_{\text{comp}}\) | Compression Ratio | Dimensionless | Uncompressed size divided by compressed size. A compressed payload has \(1/r_{\text{comp}}\) of the uncompressed size; the subscript avoids bare \(C\) collisions. |
| \(Q_{\text{req}}\) | Request Concurrency | requests | Average in-flight requests in a stable serving system (\(Q_{\text{req}} = \lambda_{\text{arr}} \cdot T_{\text{lat}}\) via Little’s Law). Avoids collision with \(N\) as device count in distributed settings. |
| \(\lambda_{\text{arr}}\) | Arrival Rate | requests/s | Request arrival rate. The subscript avoids collision with \(\lambda\) as degradation sensitivity or failure rate. |
| \(T_{\text{lat}}\) | Request Time in System | seconds | End-to-end queueing/serving latency for Little’s Law. Distinct from \(L_{\text{lat}}\), the fixed-latency term in the iron law. |
| \(T_{\text{svc}}(B)\) | Batch Service Time | seconds | Time to serve a batch of size \(B\). |
| \(\mu_{\text{eff}}(B)\) | Effective Service Rate | requests/s | Batched service rate, typically \(\mu_{\text{eff}}(B) = B/T_{\text{svc}}(B)\). |
| \(\rho_{\text{serv}}\) | Serving Utilization | Dimensionless | Queue/server utilization. Use instead of bare \(\rho\), which is reserved for communication-computation ratio in distributed contexts. |
| \(M_{\text{total}}\) | Total Memory Footprint | bytes | Sum of explicit memory components; avoids bare \(M\) ambiguity. |
| \(M_{\text{weights}}\) | Weight Memory | bytes | Memory occupied by model parameters. |
| \(M_{\text{gradients}}\) | Gradient Memory | bytes | Memory occupied by stored gradients. |
| \(M_{\text{optimizer}}\) | Optimizer-State Memory | bytes | Momentum, variance, master weights, and related optimizer buffers. |
| \(M_{\text{activations}}\) | Activation Memory | bytes | Retained activations for the backward pass or serving intermediates. |
| \(s_{\text{elem}}\) | Element Storage Size | bytes/element | Bytes per stored tensor element. |
Units and Precision
- Physical units: This book uses SI (metric) units throughout, including meters, kilograms, seconds, watts, and °C, consistent with standard engineering and scientific practice. Where source data was originally reported in imperial units, the book converts to SI and notes the original values parenthetically. A space always separates the number from the unit (for example, 100 ms, 2 TB/s).
- Data and memory: This book uses decimal SI prefixes only: KB = \(10^3\) bytes, MB = \(10^6\), GB = \(10^9\), TB = \(10^{12}\). Binary-prefixed units do not appear in prose; all capacities, throughputs, and model sizes are reported in decimal units (for example, 80 GB, 2 TB/s, 102 MB).
- Compute: The notation distinguishes operation counts from rates. Total work uses FLOPs (for example, GFLOPs, TFLOPs), while throughput uses FLOP/s with decimal prefixes (for example, GFLOP/s, TFLOP/s).
- 1 TFLOP = \(10^{12}\) FLOPs
- 1 TFLOP/s = \(10^{12}\) FLOPs per second
- Arithmetic intensity conventionally uses FLOP/byte as a unit ratio (floating-point operations per byte moved). This is a ratio unit, not a total-work symbol or throughput symbol.
- Currency: Dollar amounts use the dollar sign (
$); unless otherwise noted, dollar-denominated costs are U.S. dollars (USD). - Precision:
- FP64: Double precision (8 bytes)
- FP32: Single precision (4 bytes)
- TF32: TensorFloat-32 (19-bit Tensor Core compute mode; inputs and storage remain FP32)
- FP16: Half precision (2 bytes, standard range)
- BF16: Brain float (2 bytes, wide dynamic range)
- FP8: Quarter precision (1 byte, E4M3 or E5M2 format)
- FP4: 4-bit floating-point format
- INT8: 8-bit integer (1 byte)
- INT4: 4-bit integer; lower integer precisions follow the same uppercase
INTnpattern (for example, INT3, INT2)
Quick Reference: Resolving Collisions
Common collision points in ML Systems literature include:
| Symbol | ML Meaning | Systems Meaning | Book Convention |
|---|---|---|---|
| \(B\) | Batch Size | Bandwidth | Batch Size. Use \(\text{BW}\) for bandwidth. |
| \(P\) | Parameters | Peak FLOP/s | Parameters. Use \(R_{\text{peak}}\) for peak rate. |
| \(D\) | Dataset Size | Data Volume | Dataset Size. Use \(D_{\text{vol}}\) for bytes moved. |
| \(L\) | Loss | Latency | Loss \((\mathcal{L})\). Use \(L_{\text{lat}}\) for latency. |
| \(\eta\) | Learning Rate | Efficiency | Learning Rate. Use \(\eta_{\text{hw}}\) for efficiency. |
As a general principle, ML conventions take precedence for single letters, while systems concepts get subscripts or multi-letter symbols. This reflects the primary audience (ML practitioners learning systems) and preserves compatibility with the vast ML literature.
分布式系统记号
分布式系统工作引入了一层用于协同、通信和可靠性的记号。下表中的部分符号专用于同步、修复和梯度规模语境；集中定义它们，使得这些符号出现时规范形式无歧义。核心方程是 集群定律（分布式步时间定律）：
\[T_{\text{step}}(N) = \frac{T_{\text{compute}}}{N} + T_{\text{comm}}(N) + T_{\text{sync}}(N) - T_{\text{overlap}}\]
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(N\) | 设备数量 | Integer | 分布式任务中的加速器数量。(不是参数，使用 \(P\).) |
| \(T_{\text{step}}(N)\) | 分布式步时间 | 秒 | 在规模 \(N\) 下执行一次训练步的墙钟时间。 |
| \(T_{\text{compute}}\) | 单设备计算时间 | 秒 | 单个设备上的前向 + 反向传递。 |
| \(T_{\text{comm}}(N)\) | 通信时间 | 秒 | 集合操作（AllReduce、AllGather）所需时间。随 \(N\) 增长。 |
| \(T_{\text{overlap}}\) | 重叠时间 | 秒 | 隐藏在计算背后的通信。降低有效开销。 |
| \(T_{\text{sync}}(N)\) | 同步时间 | 秒 | 在规模 \(N\) 下每步的总非重叠同步或协同成本。 |
| \(\rho\) | 通信—计算比 | 无量纲 | \(\rho = T_{\text{comm}}(N)/(T_{\text{compute}}/N)\)。接近或高于 1 的值表示通信受限的扩展。 |
通信模型
网络通信时间分解为固定延迟和带宽相关的传输：
\[T(n) = \alpha + \frac{n}{\beta}\]
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(T(n)\) | 通信时间 | 秒 | 传输大小为 \(n\) 的消息经过一条链路的墙钟时间。 |
| \(\alpha\) | 网络延迟 | 秒 | 每条消息的固定开销。(不是学习率，上下文可消歧。) |
| \(\beta\) | 链路带宽 | 字节/秒 | 每条链路的有效吞吐量。 |
| \(n\) | 消息大小 | 字节 | 载荷大小（例如，梯度张量）。 |
| \(n^*\) | 交叉点 | 字节 | \(n^* = \alpha \cdot \beta\)。低于 \(n^*\)：延迟受限。高于：带宽受限。 |
LogGP 扩展
LogGP 通过分离网络延迟、处理器开销和长消息注入间隙来细化 \(\alpha\)-\(\beta\) 模型。标准名称较短，但本书对碰撞概率最高的项使用描述性子下标。
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(T_{\text{long}}(n)\) | LogGP 长消息时间 | 秒 | 传输一条 \(n\) 字节长消息的时间。复用 \(n\) 中的 \(\alpha\)-\(\beta\) 模型。 |
| \(L_{\text{LogGP}}\) | LogGP 网络延迟 | 秒 | 标准 LogGP 延迟参数；下标以避免与裸 \(L\) 冲突。 |
| \(o_{\text{send}}\) | 发送方开销 | 秒 | 处理器/NIC 发起一条消息的开销。 |
| \(o_{\text{recv}}\) | 接收方开销 | 秒 | 处理器/NIC 完成一条消息的开销。 |
| \(g\) | 消息注入间隙 | 秒 | 连续两次消息注入之间的最小间隔。 |
| \(G\) | 每字节长消息间隙 | 秒/字节 | 每字节长消息间隙；不同于 \(\beta\) 链路带宽。 |
扩展效率
\[\eta_{\text{scaling}} = \frac{T_1}{N \times T_N}\]
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(\eta_{\text{scaling}}\) | 扩展效率 | 无量纲 | 实测加速比与理想线性加速比之比。\(1.0\) 是理想线性基线，而非硬性上界。 |
| \(T_1\) | 单设备时间 | 秒 | 在单个设备上完成相同有效工作的基线墙钟时间。 |
| \(T_N\) | N 设备时间 | 秒 | 在 \(N\) 个设备上完成相同有效工作的墙钟时间；排除由 \(\eta_{\text{goodput}}\) 计量的运行停机时间。 |
容错与可靠性
系统可靠性随规模下降。假设组件相互独立且服从指数故障分布，跨 \(N\) 个组件的系统可靠性为：
\[R_{\text{system}}(t) = e^{-N\lambda t}\]
聚合指数使用系统级速率 \(N\lambda\)，其中 \(\lambda\) 是在独立同构组件假设下的单组件故障率。指数 \(N\lambda t\) 无量纲，因此 \(t\) 必须使用与 \(\lambda\)（当 \(\lambda\) 来自下方 FIT 推导出的速率时为小时）相同的时间单位。
最优检查点间隔在 I/O 成本与返工成本之间取得平衡：
\[\tau_{\text{opt}} = \sqrt{2 \cdot T_{\text{write}} \cdot \text{MTBF}_{\text{system}}}\]
The formula uses 系统级平均无故障时间（MTBF）（整个任务的故障间隔），而非组件级MTBF；代入单组件值会高估 \(\tau_{\text{opt}}\) 达 \(\sqrt{N}\)。为量纲一致性，\(T_{\text{write}}\) 与 \(\text{MTBF}_{\text{system}}\) 必须使用相同的时间单位。行业标准 \(\text{MTBF}\) 以小时报告；为与秒量级的写入时间合并，需转换：\(\text{MTBF}_s = \text{MTBF}_h \times 3600\)。结果 \(\tau_{\text{opt}}\) 因而带有与输入所选相同的时间单位。
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(R_{\text{system}}(t)\) | 可靠性函数 | 概率 | 在时间 \(t\) 之前整个系统不发生故障的概率。 |
| \(\lambda\) | 单组件故障率 | 故障/小时 | 单组件故障率；在独立同构组件假设下，系统级速率为 \(N\lambda\)。行业标准报告单位为 FIT（每 \(10^9\) 设备小时的故障数）：\(\lambda_{\text{hour}} = \text{FIT} \times 10^{-9}\)。(不是敏感性——上下文可消歧。) |
| \(\text{MTBF}\) | 平均无故障时间 | 小时 | 连续两次故障之间的平均时间。\(\text{MTBF}_{\text{system}} = \text{MTBF}_{\text{component}} / N\)。 |
| \(\text{MTTF}\) | 平均失效时间 | 小时 | 由 FIT 速率推导出的组件寿命指标。当建模修复/更换周期时，与系统级 \(\text{MTBF}\) 不同。 |
| \(\text{MTTR}\) | 平均修复时间 | 小时 | 故障后的平均恢复时间。 |
| \(\tau_{\text{opt}}\) | 最优检查点间隔 | 秒 | Young-Daly 公式。最小化总浪费时间。 |
| \(T_{\text{write}}\) | 检查点写入时间 | 秒 | 将模型状态持久化到存储所需的时间。 |
并行维度
三维并行性在三个正交维度上划分训练工作负载：
\[N_{\text{total}} = d \times p \times t\]
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(N_{\text{total}}\) | 设备总数 | 整数 | 三个并行维度之积。整个任务中的加速器总数。 |
| \(d\) | 数据并行 | 整数 | 模型副本数量。(也是隐藏维度，上下文可消歧。) |
| \(p\) | 物理流水线阶段 | 整数 | 模型深度划分中的物理流水线阶段数量。 |
| \(t\) | 张量并行 | 整数 | 层内划分（模型宽度）的度。(也是时间，上下文可消歧。) |
| \(m\) | 微批次数量 | 整数 | 用于填充并排空流水线的微批次数量。 |
| \(b_{\text{pipe}}\) | 流水线气泡占比 | 无量纲 | 流水线 GPU 时间中空闲所占比例；避免与裸 \(b\) 冲突。 |
| \(V\) | 每设备虚拟流水线阶段 | 整数 | 分配给每个物理流水线阶段的交错虚拟块数量。 |
| \(M\) | 消息载荷大小 | 字节 | 需通信的梯度、模型状态、激活或压缩消息载荷的总大小。 |
集群容量因子
集群级容量计算将单设备模型效率、有效步的扩展效率，以及在这些步骤之外测得的运行吞吐相乘。
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(f_{\text{compute}}\) | 有效计算时间占比 | 无量纲 | \((T_{\text{compute}}/N)/T_{\text{step}}(N)\)。衡量分布式后每步时间中有多少属于有用的本地算术。 |
| \(f_{\text{overhead}}\) | 运行开销占比 | 无量纲 | 在由 \(T_N\) 度量的有效步骤之外损失的已分配墙钟时间占比，例如检查点暂停、故障恢复、调度器抢占或维护。 |
| \(\eta_{\text{goodput}}\) | 吞吐率 | 无量纲 | 在 \(f_{\text{overhead}}\) 中的步骤外损失之后剩余的已分配墙钟时间占比；通常为 \(\eta_{\text{goodput}} = 1 - f_{\text{overhead}}\)。 |
| \(R_{\text{eff}}\) | 有效集群吞吐量 | FLOP/s | \(R_{\text{eff}} = (N R_{\text{peak}}) \times \text{MFU} \times \eta_{\text{scaling}} \times \eta_{\text{goodput}}\)，其中 \(\text{MFU}\) 为单设备基线，\(\eta_{\text{scaling}}\) 涵盖全部步内扩展损失，\(\eta_{\text{goodput}}\) 仅涵盖被 \(T_N\) 排除的步骤外运行损失。 |
压缩经济性
压缩模型将编解码器开销与通过减少载荷所节省的通信时间相比较。
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(T_{\text{transfer}}(x)\) | 载荷 \(x\) 的传输时间 | 秒 | 在相关链路或集合模型下传输 \(x\) 字节载荷的通信时间。 |
| \(T_{\text{compressed}}\) | 压缩后通信时间 | 秒 | 编码、\(M/r_{\text{comp}}\) 传输和解码后的总时间。 |
| \(T_{\text{encode}}\) | 编解码器编码时间 | 秒 | 传输前的本地压缩开销。 |
| \(T_{\text{decode}}\) | 编解码器解码时间 | 秒 | 传输后的本地解压开销。 |
大规模服务内存
| 符号 | 定义 | 单位 | 注释 |
|---|---|---|---|
| \(M_{\text{KV}}\) | KV 缓存内存 | 字节 | 自回归服务用的键值缓存占用。 |
其他符号冲突点
分布式语境在基础记号所注记的冲突之外引入了进一步的符号冲突：
| 符号 | 机器学习含义 | 分布式含义 | 我们的约定 |
|---|---|---|---|
| \(N\) | — | 设备数量 | 分布式语境下的设备数（加速器，而非主机节点）。 |
| \(\lambda\) | 敏感度 | 故障率 | 语境依赖。退化语境下为敏感度；可靠性语境下为故障率。 |
| \(d\) | 隐藏维度 | 数据并行度 | 语境依赖。3D 记号下为并行度；架构中为隐藏维度。 |
| \(t\) | — | 张量并行/时间 | 语境依赖。3D 记号下为张量并行；可靠性语境下为时间。 |
| \(\tau\) | 漂移阈值 | 本地区间/延迟 | 避免裸复用。通用检查点区间使用 \(\tau_{\text{ckpt}}\)，Young-Daly 最优值使用 \(\tau_{\text{opt}}\)，梯度陈旧度使用 \(\tau_{\text{stale}}\)。 |
其他单位
- 网络吞吐量：依据约定，分别以字节/秒（GB/s、TB/s）和比特/秒（Gbps）报告。InfiniBand 与 Ethernet 规范使用 Gbps；应用级吞吐量使用 GB/s。首次使用时我们注明约定。
- 可持续性语境下的功耗：当所在章节讨论功率或能量时，物理功耗可能使用显式下标 \(P_{\text{...}}\) 的变量（例如 \(P_{\text{IT}}\)、\(P_{\text{dynamic}}\)、\(P_{\text{average}}\)）。裸 \(P\) 保留用于参数数量。