From 4aef8db36ef48ceefceeda17064dc5144d9ef037 Mon Sep 17 00:00:00 2001 From: TheM14 Date: Tue, 3 Nov 2026 13:02:41 +0800 Subject: [PATCH] Implement PLIIx and Lo/Ro folding for warp modes 2/3 - Render 5.1-family and 4.0 layouts through expanded surround geometry before folding. - Keep trim, balance and automatic position compensation tied to the final target layout. - Preserve height channels, LFE passthrough and per-object trim bypass behavior. - Update the bilingual README and mathematical documentation. --- README.en.md | 2 +- README.md | 2 +- docs/math.en.md | 49 +++++++++++++++++++++++--- docs/math.md | 40 ++++++++++++++++++--- src/speaker_renderer.py | 77 ++++++++++++++++++++++++++++++++++++++--- src/trim_balance.py | 12 +++---- 6 files changed, 160 insertions(+), 22 deletions(-) diff --git a/README.en.md b/README.en.md index 33abde7..7a4f311 100644 --- a/README.en.md +++ b/README.en.md @@ -312,7 +312,7 @@ See the [mathematical notes](docs/math.en.md) for the equations used by the deco - The extra pair's identity follows TS 103 420 Table 47/Table 48 ($k=1\to$Lb/Rb, $k\in\{2,4\}\to$Tfl/Tfr, $N=5,7,7,5,7$); Annex E.3 requires a dependent-substream `chanmap`, when present, to indicate one of the Table 47 configurations, so `*_5_1_2top_*` (chanmap says Lts/Rts) does not satisfy that "shall". Such material is tolerated with a diagnostic instead of being refused. - An audio frame with no JOC EMDF at all (for example when the encoder wrote none for the first frame) emits silent object channels and is recorded in the `[diagnostics]` line and the JSON report's `diagnostics` field. - The speaker and SOFA binaural paths currently cover ordinary point objects; extent, spread, diffuse, divergence, channel lock, and similar controls are outside the supported scope. -- Speaker rendering reads warp 0/1, trims and front/back balance from OAMD, with no CLI warp override. Modes 2/3 are explicitly unsupported. Source trajectories and binaural remain free of these speaker controls; see [mathematics §12](docs/math.en.md) for parameters and evaluation order. +- Speaker rendering reads warp, trims and front/back balance from OAMD, with no CLI warp override. Generic rendering parameters support warp 0/1 and modes 2/3, which render to 7.1.x before PLIIx or Standard (Lo/Ro) folding to 5.1.x; calibration always follows the final target layout. Source trajectories and binaural do not bake in these speaker controls; see [mathematics §12](docs/math.en.md) for parameters and evaluation order. - Multi-data-point streams, uncommon band configurations, and unusual OAMD scheduling have less coverage than common 12-band, single-data-point material. - `joc_clipgain` is folded into the **analysis-side scalar** (`scalar = 1/16 x g`, with `g` from the `x:3 | y:5` field), so the x16 at the output cancels the 1/16 and the clamp sits **after** the gain. diff --git a/README.md b/README.md index acab2d5..acc8205 100644 --- a/README.md +++ b/README.md @@ -274,7 +274,7 @@ JustOneCacophony/ - 额外那一对的身份以 TS 103 420 Table 47/Table 48 为准($k=1\to$Lb/Rb,$k\in\{2,4\}\to$Tfl/Tfr,$N=5,7,7,5,7$);Annex E.3 规定依赖子流的 `chanmap`(若存在)**应**指明 Table 47 的某种配置,因此 `*_5_1_2top_*`(chanmap 写 Lts/Rts)属于不满足 "shall" 的素材,我们按"记诊断 + 继续解码"容错处理。 - 音频帧内没有任何 JOC EMDF 时(例如编码器未写首帧元数据),该帧的对象通道静音,并在 stderr 的 `[diagnostics]` 与 JSON 报告的 `diagnostics` 里记一笔。 - 扬声器与 SOFA 双耳路径当前只覆盖普通点对象;extent、spread、diffuse、divergence、channel lock 等对象控制不在支持范围内。 -- 扬声器渲染从 OAMD 元数据读取 warp 0/1、trim 和 front/back balance;命令行不提供 warp 覆盖。warp 2/3 暂未实现,遇到时明确报错。原始对象轨迹与 binaural 不预烘焙这些扬声器控制,参数和顺序见 [数学说明 §12](docs/math.md)。 +- 扬声器渲染从 OAMD 元数据读取 warp、trim 和 front/back balance;命令行不提供 warp 覆盖。通用渲染参数支持 warp 0/1,以及先渲染到 7.1.x 再按 PLIIx 或 Standard(Lo/Ro)折回 5.1.x 的 warp 2/3;校准参数始终按最终目标布局选择。原始对象轨迹与 binaural 不预处理这些扬声器控制,参数和顺序见 [数学说明 §12](docs/math.md)。 - 多数据点、少见参数带配置和特殊 OAMD 调度的覆盖度低于常见 12-band、单数据点素材。 - `joc_clipgain` 在**分析端**并入分析标量:`标量 = 1/16 × g`(`g = 1 + (y/32)·2^(x−4)`),出口的 ×16 抵消 1/16,净效果是 `×g`,clamp 因此处在增益**之后**。标量按帧保存两个槽(上一帧值 → 本帧值), diff --git a/docs/math.en.md b/docs/math.en.md index 3e5feb4..1ee2238 100644 --- a/docs/math.en.md +++ b/docs/math.en.md @@ -454,8 +454,8 @@ v_{\mathrm{floor}}= $$ Warp 0 and other layouts use $v_{\mathrm{floor}}=v$. The height layer is not scaled. -An absent warp indication uses 0; no CLI override is provided. Modes 2/3 are -explicitly unsupported rather than mapped to 0/1. Balance follows this scaling. +An absent warp indication uses 0; no CLI override is provided. Modes 2/3 use +the intermediate layout and folding matrix in §12.5. Balance follows this scaling. ### 10.3 Height layer @@ -563,7 +563,7 @@ Warp, trim and balance controls are read from OAMD metadata: | Parameter | Value or unit | Behavior | |---|---|---| -| `warp_mode` | 0 / 1 | Selects the floor Y mapping in §10.2; modes 2/3 are unsupported and rejected | +| `warp_mode` | 0 / 1 / 2 / 3 | Direct rendering / floor Y scaling / PLIIx folding / Standard (Lo/Ro) folding | | `global_trim_mode` | 0 / 1 / 2 | Automatic / disabled / per-layout configuration; reserved value 3 is rejected | | $T_s$, $T_h$ | dB | Explicit surround and height trim, with positive values treated as zero | | $T_c$ | dB | Signed explicit centre trim | @@ -642,7 +642,7 @@ $1$ at the back wall; $x$ and $z$ are unchanged. ### 12.4 Where it is applied Let $B_z(y)$ denote the balance map of §12.3 and $W(y)$ the warp map of §10.2. -The ordinary point-object path keeps three coordinates: +Direct point rendering in warp modes 0/1 keeps three coordinates: $$ y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y). @@ -664,6 +664,47 @@ Height and trim coordinates pass through balance alone. The trim scalar multiplies the object's entire panning vector before accumulation into output channels. LFE passes through. +### 12.5 Surround folding in warp modes 2/3 + +For a 5.1, 5.1.2 or 5.1.4 target, warp modes 2/3 first render objects to the +corresponding 7.1, 7.1.2 or 7.1.4 intermediate layout, then fold the rear +surrounds into the target surround pair. The 4.0 layout also adds a rear pair +without adding centre or LFE. Other existing layouts render directly. + +The intermediate layout changes speaker geometry and panning only. The trim +configuration, both balance layers and the maximum automatic position +compensation remain those of the final target. For example, 5.1.2 uses +configuration 4 and a maximum compensation of $-2.25$ dB even when its +intermediate layout is 7.1.2. With separate side and rear pairs in the +intermediate layout, floor Y is not doubled. Balance and per-object trim +are evaluated during intermediate rendering. + +For intermediate outputs $L_s,R_s,L_{rs},R_{rs}$ and target surrounds +$L'_s,R'_s$: + +$$ +L'_s=L_s+aL_{rs}+bR_{rs},\qquad +R'_s=R_s+bL_{rs}+aR_{rs}. +$$ + +| Mode | $a$ | $b$ | +|---|---|---| +| 2: PLIIx | $\sqrt{3}/2$ | $1/2$ | +| 3: Standard (Lo/Ro) | $1$ | $0$ | + +Both matrices add linear amplitudes without normalizing energy after the +fold. Shared front, height and LFE channels pass through to their matching +positions. The folded object gain vector enters the target-gain update and +ramp in §13 before object audio is accumulated. + +The complete order is: select calibration for the target layout → apply +balance and trim while panning in the intermediate layout → fold horizontal +surrounds → update gain ramps and mix. Per-object bypass disables balance +and trim while retaining layout selection and folding. These modes are +generic speaker-rendering controls; the publicly defined E-AC-3 JOC warp +values remain 0/1. Source trajectories, ADM and binaural do not bake in the +folded result. + ## 13. OAMD time alignment and gain ramps The coded position of an OAMD update is diff --git a/docs/math.md b/docs/math.md index ccf004c..4b01e51 100644 --- a/docs/math.md +++ b/docs/math.md @@ -451,7 +451,7 @@ v_{\mathrm{floor}}= $$ `warp_mode=0` 或其他布局使用 $v_{\mathrm{floor}}=v$。高度层不做该缩放。 -未携带 warp 元数据时采用 0;命令行不提供覆盖选项。2/3 保留为未实现,报错而不回退。 +未携带 warp 元数据时采用 0;命令行不提供覆盖选项。warp 2/3 使用 §12.5 的中间布局与折叠矩阵。 有 front/back balance 时,先求这里的分支坐标,再应用 §12.3 的 balance。 ### 10.3 高度层 @@ -557,7 +557,7 @@ warp、trim 和 balance 均从 OAMD 元数据读取。各参数的取值与作 | 参数 | 取值或单位 | 行为 | |---|---|---| -| `warp_mode` | 0 / 1 | 选择 §10.2 的地面 Y 映射;2/3 暂不支持,遇到时明确报错 | +| `warp_mode` | 0 / 1 / 2 / 3 | 直接渲染 / 地面 Y 缩放 / PLIIx 折叠 / Standard(Lo/Ro)折叠 | | `global_trim_mode` | 0 / 1 / 2 | automatic / disabled / 按目标布局配置选择;3 为保留值,明确报错 | | $T_s$、$T_h$ | dB | 显式 surround、height trim 参数,正值按 0 处理 | | $T_c$ | dB | 显式 centre trim 参数,保留符号 | @@ -628,7 +628,7 @@ $$ ### 12.4 施加位置 记 §12.3 的高度混合 balance 映射为 $B_z(y)$,§10.2 的 warp 映射为 $W(y)$。 -普通点对象的实际分支顺序为 +warp 0/1 直接渲染点对象时,分支顺序为 $$ y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y). @@ -642,11 +642,43 @@ $$ 因此可概括为 warp → balance → trim 与声像增益合成,但不能让所有阶段共用一个 已经 warp 的 Y。balance 仅在 explicit 且未逐对象 bypass 时启用;warp 不受 trim -状态或 trim bypass 控制。源坐标保持原值,ADM 与 binaural 不预烘焙此变换。 +状态或 trim bypass 控制。源坐标保持原值,ADM 与 binaural 不预处理此变换。 地面坐标依次经过 warp、balance,再用于地面声像;顶部坐标和 trim 求值坐标只经过 balance。trim 标量乘到该对象的整条声像增益向量后,再累加到输出声道。LFE 直接通过。 +### 12.5 warp 2/3 的环绕折叠 + +目标为 5.1、5.1.2 或 5.1.4 时,warp 2/3 先把对象渲染到对应的 7.1、7.1.2 或 +7.1.4 中间布局,再将后环绕折叠到目标的环绕对。4.0 同样先增加一对后环绕, +中间布局不增加 centre 或 LFE。其他现有布局直接渲染。 + +中间布局只改变扬声器几何与声像分配。Trim 配置、两层 Balance 和 Auto Trim 的 +最大位置补偿仍按最终目标布局选择。例如,5.1.2 使用配置 4,最大位置补偿为 +$-2.25$ dB,即使其中间布局是 7.1.2,也继续使用这些参数。中间布局已有独立 +side/rear 两对,地面 Y 不再乘 2;Balance 和逐对象 Trim 在中间渲染时求值。 + +设中间布局输出为 $L_s,R_s,L_{rs},R_{rs}$,最终目标的环绕输出为 $L'_s,R'_s$: + +$$ +L'_s=L_s+aL_{rs}+bR_{rs},\qquad +R'_s=R_s+bL_{rs}+aR_{rs}. +$$ + +| 模式 | $a$ | $b$ | +|---|---|---| +| 2:PLIIx | $\sqrt{3}/2$ | $1/2$ | +| 3:Standard(Lo/Ro) | $1$ | $0$ | + +两种矩阵都按线性幅度相加,不做折叠后的能量归一化。共同的前方、高度和 LFE +声道按对应位置直接通过。折叠后的对象增益向量进入 §13 的目标增益更新与斜坡, +然后累加对象音频。 + +完整顺序为:按目标布局选择校准参数 → 在中间布局应用 Balance、Trim 并计算声像 +增益 → 折叠水平环绕 → 更新增益斜坡与混音。逐对象 bypass 关闭 Balance 和 Trim, +但仍保留 warp 2/3 的布局选择与折叠。上述模式属于通用扬声器渲染控制;E-AC-3 JOC +公开定义的 warp 值仍为 0/1,源对象轨迹、ADM 和 binaural 不预处理折叠结果。 + ## 13. OAMD 时间对齐与增益斜坡 OAMD 更新的编码位置为 diff --git a/src/speaker_renderer.py b/src/speaker_renderer.py index 72153b5..6a113af 100644 --- a/src/speaker_renderer.py +++ b/src/speaker_renderer.py @@ -7,6 +7,7 @@ writes a lower-precision output format. from __future__ import annotations from collections import defaultdict +from functools import lru_cache import math from typing import Iterable @@ -182,11 +183,81 @@ def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float """Render one point object to a target speaker layout using float64.""" target = get_speaker_layout(layout) if isinstance(layout, str) else layout controls = parameters or parameters_from_oamd(None, target) + source = _surround_fold_layout(target) if controls.warp_mode in (2, 3) else target + # Geometry may expand to 7.1.x; calibration always uses the final layout. + internal = _render_point_gains(source, u, v, w, region_index=region_index, + enable_height=enable_height, object_gain=object_gain, + parameters=controls, bypass_trim=bypass_trim, + compensation_bitfield=target.speaker_bitfield) + if source is not target: + internal = _surround_fold_matrix(source.speaker_bitfield, target.speaker_bitfield, + controls.warp_mode) @ internal + if standard_order: + return internal[list(target.internal_to_standard)] + return internal + + +@lru_cache(maxsize=16) +def _surround_fold_layout(target: SpeakerLayout) -> SpeakerLayout: + """Add a rear pair for one-pair surround layouts, keeping the height layer.""" + if not floor_y_exponent(target.speaker_bitfield): + return target + expanded = {"51": "71", "512": "712", "514": "714"} + if target.name in expanded: + return get_speaker_layout(expanded[target.name]) + if target.name != "40": + raise ValueError(f"unsupported surround fold layout: {target.name}") + # 4.0 expands to six full-band speakers without adding centre or LFE. + base = get_speaker_layout("71") + regions = [] + speaker_ids = {0: 0, 1: 1, 4: 2, 5: 3, 6: 4, 7: 5} + for region in base.regions: + kept = [i for i, speaker in enumerate(region.speaker_ids) if speaker in speaker_ids] + indices = {old: new for new, old in enumerate(kept)} + def groups(items): + return tuple(tuple(indices[i] for i in group if i in indices) + for group in items if any(i in indices for i in group)) + regions.append(RegionGeometry( + tuple(region.coordinates_q15[i] for i in kept), + tuple(speaker_ids[region.speaker_ids[i]] for i in kept), + groups(region.axis0_groups), groups(region.axis1_groups), region.mode)) + return SpeakerLayout("60", 255, target.speaker_bitfield | 0x10, + ("L", "R", "Ls", "Rs", "Lrs", "Rrs"), + tuple(range(6)), tuple(regions)) + + +@lru_cache(maxsize=32) +def _surround_fold_matrix(source_bitfield: int, target_bitfield: int, + warp_mode: int) -> np.ndarray: + """Copy shared speakers and fold the rear pair into the target surround pair.""" + def speakers(bitfield): + mask = expand_speaker_bitfield(bitfield) + return tuple(i for i in range(mask.bit_length()) if mask & (1 << i)) + source, target = speakers(source_bitfield), speakers(target_bitfield) + matrix = np.zeros((len(target), len(source)), dtype=np.float64) + same_side, cross_side = (math.sqrt(3.0) / 2.0, 0.5) if warp_mode == 2 else (1.0, 0.0) + # Speaker IDs 4/5 are side surrounds; 6/7 are the rear pair. + for column, speaker in enumerate(source): + if speaker in target: + matrix[target.index(speaker), column] = 1.0 + elif speaker in (6, 7): + matrix[target.index(speaker - 2), column] = same_side + matrix[target.index(5 if speaker == 6 else 4), column] = cross_side + matrix.flags.writeable = False + return matrix + + +def _render_point_gains(target: SpeakerLayout, u: float, v: float, w: float, + *, region_index: int, enable_height: bool, object_gain: float, + parameters: SpeakerRenderParameters, bypass_trim: bool, + compensation_bitfield: int) -> np.ndarray: + """Evaluate point gains before any surround fold or output-channel reorder.""" + controls = parameters region = target.regions[int(region_index)] coordinates = _coordinates(region) v, w = float(v), max(float(w), 0.0) floor_v = min(max(math.ldexp(v, floor_y_exponent( - target.speaker_bitfield, controls.warp_mode == 1)), 0.0), 1.0) + target.speaker_bitfield, controls.warp_mode != 0)), 0.0), 1.0) top_v = min(max(v, 0.0), 1.0) trim_v = v if controls.trim_mode == "explicit" and not bypass_trim: @@ -210,13 +281,11 @@ def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float gain = float(object_gain) if not bypass_trim: if controls.trim_mode == "automatic": - gain *= position_gain(target.speaker_bitfield, trim_v, w) + gain *= position_gain(compensation_bitfield, trim_v, w) elif controls.trim_mode == "explicit": gain *= explicit_position_gain(controls, u, trim_v, w) for point_gain, speaker_id in zip(point_gains, region.speaker_ids): internal[speaker_id] = point_gain * gain - if standard_order: - return internal[list(target.internal_to_standard)] return internal diff --git a/src/trim_balance.py b/src/trim_balance.py index 9c435de..acd2511 100644 --- a/src/trim_balance.py +++ b/src/trim_balance.py @@ -58,8 +58,9 @@ WALL = 0.05 class SpeakerRenderParameters: """Codec-independent, per-layout speaker controls; balances are in [-1,1]. - Warp 2/3 intentionally have no rendering implementation yet. An absent - warp indication uses normal (0); no CLI override is provided. + Warp 0 renders directly, 1 scales floor Y, and 2/3 render through an + expanded surround layout before PLIIx/Lo/Ro folding. An absent warp + indication uses normal (0); no CLI override is provided. """ warp_mode: int = 0 trim_mode: str = "automatic" @@ -71,12 +72,7 @@ class SpeakerRenderParameters: disable_per_object: tuple[bool, ...] = () def __post_init__(self): - if self.warp_mode in (2, 3): - raise UnsupportedVariantError( - "speaker", "warp_mode_not_implemented", - "Speaker rendering implements warp modes 0/1 only; 2/3 are reserved in E-AC-3 JOC", - details={"warp_mode": self.warp_mode}) - if self.warp_mode not in (0, 1): + if self.warp_mode not in (0, 1, 2, 3): raise ValueError(f"invalid warp mode: {self.warp_mode!r}") if self.trim_mode not in ("automatic", "disabled", "explicit"): raise ValueError(f"invalid trim mode: {self.trim_mode!r}")