diff --git a/README.en.md b/README.en.md index e54d77c..33abde7 100644 --- a/README.en.md +++ b/README.en.md @@ -2,11 +2,11 @@ [中文版](README.md) -> JustOneCacophony is an experimental/test implementation of E-AC-3 JOC for studying JOC parsing, reconstruction, rendering, and the associated mathematics. +> JustOneCacophony implements E-AC-3 JOC parsing, object reconstruction and rendering. The project can extract and parse EMDF, ID14 JOC parameters, and ID11 OAMD metadata from common E-AC-3 JOC streams. It combines those data with the core PCM decoded by FFmpeg (N=5 or 7 full-band channels), reconstructs LFE plus 15 object channels, and writes ADM BWF, a WAV file for a selected speaker layout, or binaural stereo (a Rosella personalized model, or a compiled `.jochrtf`). -This is research code, not a complete, standards-compliant, or production-grade JOC decoder. It covers only the stream forms currently implemented. Unknown variants fail explicitly—because when the math goes wrong, all that may remain is the cacophony. +It covers the currently implemented stream forms. Unknown variants fail explicitly. ## Current features @@ -225,12 +225,12 @@ path: - `1` (default): the public bank, 13-tap hybrid / 6-slot high-band delay, 961 samples of latency; -- `2`: the Rosella bank captured from the reference renderer, 11-tap hybrid / +- `2`: the 11-tap Rosella bank, 11-tap hybrid / 3-slot high-band delay, 448 samples of latency. The option only applies to the Rosella model; the SOFA and `.jochrtf` paths reject it. See [Rosella filterbank](docs/rosella_filterbank.en.md) for the -parameters, buffers, latency and validation of both banks. +parameters, buffers and latency of both banks. For all options: @@ -238,7 +238,7 @@ For all options: python main.py --help ``` -Without `-o`, output still goes to `output/` at the repository root. The directory move intentionally preserves this behavior. +Without `-o`, output goes to `output/` at the repository root. ## Speaker layouts and channel order @@ -312,7 +312,7 @@ See the [mathematical notes](docs/math.en.md) for the equations used by the deco - The extra pair's identity follows TS 103 420 Table 47/Table 48 ($k=1\to$Lb/Rb, $k\in\{2,4\}\to$Tfl/Tfr, $N=5,7,7,5,7$); Annex E.3 requires a dependent-substream `chanmap`, when present, to indicate one of the Table 47 configurations, so `*_5_1_2top_*` (chanmap says Lts/Rts) does not satisfy that "shall". Such material is tolerated with a diagnostic instead of being refused. - An audio frame with no JOC EMDF at all (for example when the encoder wrote none for the first frame) emits silent object channels and is recorded in the `[diagnostics]` line and the JSON report's `diagnostics` field. - The speaker and SOFA binaural paths currently cover ordinary point objects; extent, spread, diffuse, divergence, channel lock, and similar controls are outside the supported scope. -- OAMD trim elements are boundary-checked and skipped; warp, balance, and trim parameters are not applied to raw object trajectories or speaker rendering. +- Speaker rendering reads warp 0/1, trims and front/back balance from OAMD, with no CLI warp override. Modes 2/3 are explicitly unsupported. Source trajectories and binaural remain free of these speaker controls; see [mathematics §12](docs/math.en.md) for parameters and evaluation order. - Multi-data-point streams, uncommon band configurations, and unusual OAMD scheduling have less coverage than common 12-band, single-data-point material. - `joc_clipgain` is folded into the **analysis-side scalar** (`scalar = 1/16 x g`, with `g` from the `x:3 | y:5` field), so the x16 at the output cancels the 1/16 and the clamp sits **after** the gain. diff --git a/README.md b/README.md index 0317f36..acab2d5 100644 --- a/README.md +++ b/README.md @@ -2,11 +2,11 @@ [English](README.en.md) -> JustOneCacophony 是一个 E-AC-3 JOC 的实验性 / 测试实现,用于研究 JOC 的解析、重建、渲染以及相关数学过程。 +> JustOneCacophony 实现 E-AC-3 JOC 的解析、对象重建与渲染。 项目可以从常见 E-AC-3 JOC 码流中提取并解析 EMDF、ID14 JOC 参数和 ID11 OAMD 元数据,结合 FFmpeg 解码出的核心 PCM(N=5 或 7 个全频段通道)重建 LFE 与 15 路对象 PCM,并输出 ADM BWF、指定扬声器布局的 WAV,或输出双耳 WAV(Rosella 个性化模型,或已编译的 `.jochrtf`)。 -这是研究代码,不是完整、标准兼容或生产级的 JOC 解码器。它只覆盖当前已实现的码流形态;遇到未知变体时会明确报错,而不是假装一切都很和谐——如果哪里算错了,它可能就真的只剩 cacophony 了。 +当前覆盖已实现的码流形态;遇到未知变体时明确报错。 ## 当前功能 @@ -193,11 +193,11 @@ outer subpayload offset 与 block offset 落到绝对 sample timeline,再加 `--rosella-filterbank 1|2` 选择 `.personalized_headphone` 路径使用的滤波器组: - `1`(默认):公开滤波器组,13-tap hybrid / 6-slot 高频延迟,961 samples 延迟; -- `2`:从参考渲染器捕获的 Rosella 滤波器组,11-tap hybrid / 3-slot 高频延迟, +- `2`:11-tap Rosella 滤波器组,11-tap hybrid / 3-slot 高频延迟, 448 samples 延迟。 该选项只作用于 Rosella 模型;SOFA 与 `.jochrtf` 路径给出时报错。两套实现的参数、 -缓冲、延迟与验证见 [Rosella 滤波器组](docs/rosella_filterbank.md)。 +缓冲与延迟见 [Rosella 滤波器组](docs/rosella_filterbank.md)。 更多参数可查看: @@ -205,7 +205,7 @@ outer subpayload offset 与 block offset 落到绝对 sample timeline,再加 python main.py --help ``` -未指定 `-o` 时,输出仍写入仓库根目录的 `output/`。这是文件移动后特意保持的原有行为。 +未指定 `-o` 时,输出写入仓库根目录的 `output/`。 ## 扬声器布局与声道顺序 @@ -274,7 +274,7 @@ JustOneCacophony/ - 额外那一对的身份以 TS 103 420 Table 47/Table 48 为准($k=1\to$Lb/Rb,$k\in\{2,4\}\to$Tfl/Tfr,$N=5,7,7,5,7$);Annex E.3 规定依赖子流的 `chanmap`(若存在)**应**指明 Table 47 的某种配置,因此 `*_5_1_2top_*`(chanmap 写 Lts/Rts)属于不满足 "shall" 的素材,我们按"记诊断 + 继续解码"容错处理。 - 音频帧内没有任何 JOC EMDF 时(例如编码器未写首帧元数据),该帧的对象通道静音,并在 stderr 的 `[diagnostics]` 与 JSON 报告的 `diagnostics` 里记一笔。 - 扬声器与 SOFA 双耳路径当前只覆盖普通点对象;extent、spread、diffuse、divergence、channel lock 等对象控制不在支持范围内。 -- OAMD trim element 会按声明边界校验并跳过;warp、balance 和 trim 参数不应用于当前原始对象轨迹或扬声器渲染。 +- 扬声器渲染从 OAMD 元数据读取 warp 0/1、trim 和 front/back balance;命令行不提供 warp 覆盖。warp 2/3 暂未实现,遇到时明确报错。原始对象轨迹与 binaural 不预烘焙这些扬声器控制,参数和顺序见 [数学说明 §12](docs/math.md)。 - 多数据点、少见参数带配置和特殊 OAMD 调度的覆盖度低于常见 12-band、单数据点素材。 - `joc_clipgain` 在**分析端**并入分析标量:`标量 = 1/16 × g`(`g = 1 + (y/32)·2^(x−4)`),出口的 ×16 抵消 1/16,净效果是 `×g`,clamp 因此处在增益**之后**。标量按帧保存两个槽(上一帧值 → 本帧值), diff --git a/data/README.en.md b/data/README.en.md index 7ca0a40..0f0b155 100644 --- a/data/README.en.md +++ b/data/README.en.md @@ -72,10 +72,9 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q The same values also appear in other public implementations of these standards (for example FFmpeg's `aacps_tablegen.h` and `aacsbrdata.h`). -### Captured Rosella filterbank +### 11-tap Rosella filterbank -`rosella_captured_kernels.npz` holds the bank the reference headphone renderer -actually runs (`--rosella-filterbank 2`): +`rosella_captured_kernels.npz` holds the bank selected by `--rosella-filterbank 2`: ```text format_version little-endian int32[1] @@ -85,11 +84,8 @@ hybrid complex64[22] ``` The table values are promoted to float64/complex128 on load. The full -analysis/synthesis latency is 448 samples (961 for the public bank). These -values come from runtime buffer captures of the reference renderer, not from the -public standards above; the C++ core compiles the same data in as -`src/hrtf/rosella_filterbank_tables.h`. This file also contains no personalized -HRTF or room tuning. +analysis/synthesis latency is 448 samples (961 for the public bank). This file +contains no personalized HRTF or room tuning. Public availability of a standard does not by itself grant permission to practice related patent claims. diff --git a/data/README.md b/data/README.md index c5824c2..cd2136b 100644 --- a/data/README.md +++ b/data/README.md @@ -64,10 +64,9 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q 相同数值可在 FFmpeg(`aacps_tablegen.h`、`aacsbrdata.h`)等公开实现中查到。 -### Rosella 捕获版滤波器组 +### 11-tap Rosella 滤波器组 -`rosella_captured_kernels.npz` 保存参考头戴渲染器实际运行的那套滤波器组 -(`--rosella-filterbank 2`): +`rosella_captured_kernels.npz` 保存 `--rosella-filterbank 2` 使用的滤波器组: ```text format_version little-endian int32[1] @@ -77,9 +76,7 @@ hybrid complex64[22] ``` 表值载入后提升为 float64/complex128;全链 latency 为 448 samples(公开版为 961)。 -这些数值来自参考渲染器的运行时缓冲捕获,不是上述公开标准的推导值;C++ 内核以 -`src/hrtf/rosella_filterbank_tables.h` 的形式内置同一组数据。文件同样不含个性化 -HRTF 或 room 调参。 +文件不含个性化 HRTF 或 room 调参。 标准可公开获取不等于获准实施相关专利。 diff --git a/docs/math.en.md b/docs/math.en.md index 4a13cf5..3e5feb4 100644 --- a/docs/math.en.md +++ b/docs/math.en.md @@ -446,14 +446,16 @@ $$ $$ For layouts with one horizontal surround pair rather than separate side and rear pairs (the 5.1 -family, and 4.0, which carries one rear pair only), the longitudinal coordinate is +family, and 4.0, which carries one rear pair only), OAMD `warp_mode=1` uses $$ v_{\mathrm{floor}}= \mathrm{clamp}(2v,0,1). $$ -Other layouts use $v_{\mathrm{floor}}=v$. +Warp 0 and other layouts use $v_{\mathrm{floor}}=v$. The height layer is not scaled. +An absent warp indication uses 0; no CLI override is provided. Modes 2/3 are +explicitly unsupported rather than mapped to 0/1. Balance follows this scaling. ### 10.3 Height layer @@ -474,6 +476,9 @@ $$ ## 11. Layout-dependent position compensation +This is the per-object gain for automatic trim. Disabled trim uses unity; +explicit trim uses §12.2 instead, without multiplying automatic compensation again. + Let $N_h$ be the number of relevant height speakers and $N_f$ the number of relevant additional horizontal speakers: $$ @@ -552,35 +557,53 @@ speakers by no / some / many height speakers". Trim and front-back balance apply to seven layouts only: 2.0, 3.1, 4.0, 5.1, 5.1.2, 5.1.4 and 7.1. Wider layouts (7.1.2, 7.1.4, 9.1.4, 9.1.6) are left -untrimmed, with $G_c=1$ and $b=0$. +untrimmed, with $g_t=1$ and $b=0$. -### 12.2 Group levels +Warp, trim and balance controls are read from OAMD metadata: -Each configuration supplies one level $T_g$ in dB for the centre, surround and -height groups. Only an explicit configuration contributes a static level: +| Parameter | Value or unit | Behavior | +|---|---|---| +| `warp_mode` | 0 / 1 | Selects the floor Y mapping in §10.2; modes 2/3 are unsupported and rejected | +| `global_trim_mode` | 0 / 1 / 2 | Automatic / disabled / per-layout configuration; reserved value 3 is rejected | +| $T_s$, $T_h$ | dB | Explicit surround and height trim, with positive values treated as zero | +| $T_c$ | dB | Signed explicit centre trim | +| $b_{\mathrm{lis}}$, $b_{\mathrm{tb}}$ | $[-1,1]$, step $1/16$ | Listener-plane and overhead front-back balance | +| Per-object trim bypass | Boolean | Disables the object's balance and trim gain while retaining warp | + +With global mode 2, the selected configuration's default or disabled flag +chooses automatic or disabled trim. A missing configuration uses automatic; +otherwise its explicit parameters apply. Automatic uses §11, disabled uses +unity, and explicit uses §12.2. These modes are mutually exclusive. + +Without an initial trim element, warp is 0 and trim is automatic. A later +payload without that element retains the previous controls; a new element +updates them at the corresponding metadata event boundary. The CLI has no +warp override. + +### 12.2 Per-object levels + +Centre, surround and height levels parameterize a position-dependent object +gain; they are not fixed gains applied to the output channels after mixing. +For original Y after balance $y_t$ and $z=\max(w,0)$: $$ -T_g= -\begin{cases} -\text{coded value}, & \text{explicit},\ -0, & \text{disabled},\ -0, & \text{automatic}. -\end{cases} +a_s=\mathrm{clamp}(2y_t,0,1)(1-z),\quad a_h=z, $$ -An automatic configuration carries no static group gain: the attenuation it -describes is already applied per object by the position compensation of §11, -and multiplying it in a second time would count it twice. - -The static gain of channel $c$ is - $$ -G_c=10^{T_{g(c)}/20}, +a_c=1-\mathrm{clamp}\bigl(10\max(|z|-0.05,y_t-0.05,|u-0.5|-0.05),0,1\bigr). $$ -where $g(c)$ follows the channel label: centre maps to centre, side, rear and -wide map to surround, top maps to height, and left, right and LFE stay at -$G_c=1$. +Explicit trim uses + +$$ +D=a_s\min(T_s,0)+a_h\min(T_h,0)+a_cT_c,\qquad g_t=10^{D/20}. +$$ + +The scalar multiplies the object's whole panning vector before accumulation. +Automatic trim uses §11; disabled trim or per-object bypass uses unity. LFE +passes through. Gain evaluation, panning interpolation and object accumulation +use float64. ### 12.3 Front-back balance @@ -618,18 +641,28 @@ $1$ at the back wall; $x$ and $z$ are unchanged. ### 12.4 Where it is applied -The balance acts on the second component of the object position before -panning: +Let $B_z(y)$ denote the balance map of §12.3 and $W(y)$ the warp map of §10.2. +The ordinary point-object path keeps three coordinates: $$ -y_o\longmapsto y_o', +y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y). $$ -while the group gain multiplies the panning result, that is $g_{o,c}$ in the -mix formula of §14 becomes $G_c\,g_{o,c}$. +Floor and height panning use $y_f,y_h$; trim evaluation uses $y_t$. The final gain is -Warp takes no part in either step: it selects a downmix mode, affects only the -downmix path, and leaves object coordinates untouched. +$$ +g_{o,c}=G_{\mathrm{object}}\,g_t(u,y_t,z)\,P_c(u,y_f,y_h,z). +$$ + +Warp precedes balance, followed by trim/pan gain composition, but not every +stage shares a warped Y. Balance requires explicit trim without object bypass; +warp remains independent of trim state and bypass. Source positions are kept +unchanged for ADM and binaural. + +Floor coordinates pass through warp and then balance before floor panning. +Height and trim coordinates pass through balance alone. The trim scalar +multiplies the object's entire panning vector before accumulation into output +channels. LFE passes through. ## 13. OAMD time alignment and gain ramps diff --git a/docs/math.md b/docs/math.md index 9151cf9..ccf004c 100644 --- a/docs/math.md +++ b/docs/math.md @@ -443,14 +443,16 @@ $$ $$ 对于只有一对水平环绕、没有独立 side/rear 两对的布局(5.1 系列,以及 4.0 这类 -只有一对后置声道的布局),纵向坐标使用 +只有一对后置声道的布局),仅在 OAMD `warp_mode=1` 时,纵向坐标使用 $$ v_{\mathrm{floor}}= \mathrm{clamp}(2v,0,1). $$ -其他布局使用 $v_{\mathrm{floor}}=v$。 +`warp_mode=0` 或其他布局使用 $v_{\mathrm{floor}}=v$。高度层不做该缩放。 +未携带 warp 元数据时采用 0;命令行不提供覆盖选项。2/3 保留为未实现,报错而不回退。 +有 front/back balance 时,先求这里的分支坐标,再应用 §12.3 的 balance。 ### 10.3 高度层 @@ -471,6 +473,9 @@ $$ ## 11. 布局位置补偿 +本节是 trim 为 automatic 时的逐对象增益。disabled 取 1;explicit 改用 §12.2, +不能把 automatic 增益与 explicit trim 再乘一遍。 + 令 $N_h$ 为相关高度扬声器数,$N_f$ 为相关附加水平扬声器数: $$ @@ -546,38 +551,54 @@ $$ 九个配置依次为“no / some / many 附加水平 × no / some / many 高度”。 trim 与前后平衡只在七种布局下施加:2.0、3.1、4.0、5.1、5.1.2、5.1.4、7.1。 -更宽的布局(7.1.2、7.1.4、9.1.4、9.1.6)不施加,取 $G_c=1$、$b=0$。 +更宽的布局(7.1.2、7.1.4、9.1.4、9.1.6)取 $g_t=1$、$b=0$。 -### 12.2 组电平 +warp、trim 和 balance 均从 OAMD 元数据读取。各参数的取值与作用为: -每个配置给 centre、surround、height 三个组各一个电平 $T_g$(dB)。只有显式配置 -施加静态电平: +| 参数 | 取值或单位 | 行为 | +|---|---|---| +| `warp_mode` | 0 / 1 | 选择 §10.2 的地面 Y 映射;2/3 暂不支持,遇到时明确报错 | +| `global_trim_mode` | 0 / 1 / 2 | automatic / disabled / 按目标布局配置选择;3 为保留值,明确报错 | +| $T_s$、$T_h$ | dB | 显式 surround、height trim 参数,正值按 0 处理 | +| $T_c$ | dB | 显式 centre trim 参数,保留符号 | +| $b_{\mathrm{lis}}$、$b_{\mathrm{tb}}$ | $[-1,1]$,步长 $1/16$ | 听音平面、顶部的前后 balance | +| 逐对象 trim bypass | 布尔值 | 关闭该对象的 balance 和 trim 增益,保留 warp | + +全局模式为 2 时,所选配置的 default 或 disabled 标志分别采用 automatic 或 disabled; +缺失配置采用 automatic,其他配置使用其显式参数。automatic 使用 §11 的位置补偿, +disabled 使用单位增益,explicit 使用 §12.2,三种模式互斥。 + +初始没有 trim element 时采用 `warp_mode=0`、automatic trim。后续 payload 不含该 +element 时保持上一组控制;出现新 element 时,在对应元数据事件边界更新。命令行不提供 +warp 覆盖选项。 + +### 12.2 逐对象电平 + +centre、surround、height 是位置相关 trim 函数的参数,不是混音后各输出声道的固定增益。 +令 $y_t$ 为原始 Y 经 balance 后的坐标,$z=\max(w,0)$: $$ -T_g= -\begin{cases} -\text{记录值}, & \text{显式},\ -0, & \text{禁用},\ -0, & \text{自动}. -\end{cases} +a_s=\mathrm{clamp}(2y_t,0,1)(1-z),\qquad a_h=z, $$ -自动配置不带静态组增益:它描述的衰减量已由 §11 的位置补偿按对象逐个施加,再乘 -一次会重复计算。 - -声道 $c$ 的静态增益为 - $$ -G_c=10^{T_{g(c)}/20}, +a_c=1-\mathrm{clamp}\bigl(10\max(|z|-0.05,\ y_t-0.05,\ |u-0.5|-0.05),0,1\bigr). $$ -$g(c)$ 由声道标签决定:中置归 centre,侧、后、宽归 surround,顶部归 height,左右 -与 LFE 不参与,取 $G_c=1$。 +explicit 状态下,surround/height 参数上限为 0 dB,centre 保留有符号电平: + +$$ +D=a_s\min(T_s,0)+a_h\min(T_h,0)+a_cT_c,\qquad g_t=10^{D/20}. +$$ + +同一个 $g_t$ 乘该对象的整条声像增益向量,然后才累加不同对象。automatic 使用 §11; +disabled 或逐对象 trim bypass 取 $g_t=1$。LFE 仍直接通过。 +增益求值、声像插值和对象累加均使用 float64。 ### 12.3 前后平衡 每个配置还携带两个前后平衡量:$b_{\mathrm{lis}}$ 作用于听音平面, -$b_{\mathrm{tb}}$ 作用于顶上层层。二者按 $1/16$ 网格量化,只在显式配置下非零。 +$b_{\mathrm{tb}}$ 作用于顶部。二者按 $1/16$ 网格量化,只在显式配置下非零。 记 $n=\mathrm{round}(16\,|b|)$、$s=n/18$,单层映射为 @@ -606,15 +627,25 @@ $$ ### 12.4 施加位置 -前后平衡在声像之前作用于对象坐标的第二个分量: +记 §12.3 的高度混合 balance 映射为 $B_z(y)$,§10.2 的 warp 映射为 $W(y)$。 +普通点对象的实际分支顺序为 $$ -y_o\longmapsto y_o', +y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y). $$ -组增益则乘在声像结果上,即 §14 混音式中的 $g_{o,c}$ 替换为 $G_c\,g_{o,c}$。 +地面和高度声像分别使用 $y_f,y_h$,trim 函数使用 $y_t$。最后组合 -warp 不参与上述任何一步:它是下混模式的选择位,只影响下混路径,不改变对象坐标。 +$$ +g_{o,c}=G_{\mathrm{object}}\,g_t(u,y_t,z)\,P_c(u,y_f,y_h,z). +$$ + +因此可概括为 warp → balance → trim 与声像增益合成,但不能让所有阶段共用一个 +已经 warp 的 Y。balance 仅在 explicit 且未逐对象 bypass 时启用;warp 不受 trim +状态或 trim bypass 控制。源坐标保持原值,ADM 与 binaural 不预烘焙此变换。 + +地面坐标依次经过 warp、balance,再用于地面声像;顶部坐标和 trim 求值坐标只经过 +balance。trim 标量乘到该对象的整条声像增益向量后,再累加到输出声道。LFE 直接通过。 ## 13. OAMD 时间对齐与增益斜坡 diff --git a/docs/rosella_filterbank.en.md b/docs/rosella_filterbank.en.md index a439ecf..1ecf283 100644 --- a/docs/rosella_filterbank.en.md +++ b/docs/rosella_filterbank.en.md @@ -3,8 +3,8 @@ [中文](rosella_filterbank.md) · [Back to README](../README.en.md) The Rosella `.personalized_headphone` path can run either of two 64-QMF / -77-hybrid filterbanks: the shared public bank (the default) or the Rosella bank -captured from the reference renderer. The SOFA and `.jochrtf` paths always use +77-hybrid filterbanks: the 13-tap public bank (the default) or the 11-tap Rosella +bank. The SOFA and `.jochrtf` paths always use the public bank and never consult the selector. ## Which bank runs @@ -26,30 +26,21 @@ and never share it. Any other value is rejected, and the option is an error when the binaural input is not a Rosella model: a SOFA or `.jochrtf` render must not silently ignore it. -## Why the banks differ +## Bank parameters -The reference headphone renderer selects its analysis and synthesis callbacks -during preparation, after initialization. Capturing only the initial callbacks -identified the wrong filterbank variant. Runtime buffer captures gave these -parameters: - -| Parameter | Captured Rosella bank | Public bank | +| Parameter | 11-tap Rosella bank | 13-tap public bank | | --- | --- | --- | | QMF prototype | 640 asymmetric coefficients | polyphase reorder of the public 640-tap prototype | | Low-band hybrid filter | 11 taps | 13 taps | | High-band hybrid delay | 3 slots | 6 slots | | Analysis/synthesis latency | 448 samples | 961 samples | -The wrong variant produced a frequency-dependent room-tail error. With the -captured room send as input, the existing room recursion already matched the -reference, so the fix changes only the filterbank: not the room coefficients, -the direct HRTF calculation, the phase mapping, or the output EQ. The public -bank stayed as bank `1` so historical renders remain reproducible; the captured -bank is bank `2`. +The selector does not change room coefficients, direct HRTF calculation, phase +mapping or output EQ. Bank `1` is the default. ## Implementation -`rosella_captured_filterbank.py` reproduces the four half-steps of the captured +`rosella_captured_filterbank.py` executes the four half-steps of the 11-tap bank in float64/complex128: - QMF analysis keeps a 9-slot ring history, reads `(position + tap + 1) mod 10`, @@ -60,30 +51,16 @@ bank in float64/complex128: - QMF synthesis keeps a 9-slot history and accumulates 10 prototype-matched lags, taking the imaginary part on odd lags. -`data/rosella_captured_kernels.npz` stores the captured prototype +`data/rosella_captured_kernels.npz` stores the 11-tap bank's prototype (`float32[64,10]`), rotation (`complex64[64]`) and hybrid coefficients -(`complex64[22]`); they are promoted to float64/complex128 on load. The values -are the ones the C++ core compiles in as `src/hrtf/rosella_filterbank_tables.h` -and contain no personalized HRTF or room tuning. +(`complex64[22]`); they are promoted to float64/complex128 on load and contain +no personalized HRTF or room tuning. The startup skip and the flush capacity both come from the selected bank (961 or 448 samples). `.report.json` records the actual choice in `binaural_renderer_backend`, as `filterbank`, `filterbank_name` and `latency_compensated_samples`. -## Validation - -The local regression script lives in `tests/`, which this repository does not -track (like the rest of its local checks). It covers: - -- reference vectors for the four captured half-steps, the same samples the C++ - core checks, generated by an independent NumPy implementation, to within - `1e-10` per sample; -- an impulse roundtrip peak at 961 and 448 samples for the two banks; -- chunk invariance (including zero-length and unequal chunks) and `reset()`; -- independence: processing one bank never changes the other bank's state; -- rejection of out-of-range values. - -The public filterbank formulas and `.jochrtf` direction field described in -`docs/binaural.md` are unchanged; the captured bank only affects the Rosella +The public filterbank formulas and `.jochrtf` direction field are described in +`docs/binaural.md`; the 11-tap bank only affects the Rosella path. diff --git a/docs/rosella_filterbank.md b/docs/rosella_filterbank.md index 325edeb..7a64e0e 100644 --- a/docs/rosella_filterbank.md +++ b/docs/rosella_filterbank.md @@ -3,7 +3,7 @@ [English](rosella_filterbank.en.md) · [返回 README](../README.md) Rosella `.personalized_headphone` 路径可以在两套 64-QMF / 77-hybrid 滤波器组之间选择: -公开滤波器组(默认)与从参考渲染器捕获的 Rosella 滤波器组。SOFA 与 `.jochrtf` 路径 +13-tap 公开滤波器组(默认)与 11-tap Rosella 滤波器组。SOFA 与 `.jochrtf` 路径 始终使用公开滤波器组,不经过该选择。 ## 选择哪一套 @@ -21,25 +21,20 @@ QMF 合成),因此切换会同时改变系数、高频延迟、历史缓冲 flush 容量;分析与合成两级各自持有状态,互不共享。其它取值直接报错;非 Rosella 输入(SOFA、`.jochrtf`)给出该选项也报错,不会静默忽略。 -## 为什么两套不同 +## 两套参数 -参考头戴渲染器在初始化之后的准备阶段才选定 analysis/synthesis 回调,只捕获初始化 -时的回调会拿到错误的滤波器组变体。运行时缓冲捕获得到的参数是: - -| 参数 | Rosella 捕获版 | 公开版 | +| 参数 | 11-tap Rosella | 13-tap 公开版 | | --- | --- | --- | | QMF prototype | 640 个非对称系数 | 公开 640-tap prototype 的多相重排 | | 低频 hybrid 滤波 | 11 taps | 13 taps | | 高频 hybrid delay | 3 slots | 6 slots | | analysis/synthesis 延迟 | 448 samples | 961 samples | -用错变体会在 room 尾音上引入随频率变化的误差;把捕获到的 room send 作为输入时, -现有 room 递推已经能对上参考,因此修复只更换滤波器组,不改 room 系数、direct HRTF、 -相位映射或输出 EQ。公开版是 `1`,捕获版是 `2`;默认保留公开版,便于与历史输出对齐。 +该选项不改变 room 系数、direct HRTF、相位映射或输出 EQ。默认选择 `1`。 ## 实现 -`rosella_captured_filterbank.py` 用 float64/complex128 复现捕获版的四个 half-step: +`rosella_captured_filterbank.py` 用 float64/complex128 执行 11-tap 滤波器组的四个 half-step: - QMF 分析保留 9 slot 环形历史,按 `(position + tap + 1) mod 10` 读取,输出 `spectrum[2b] = F[b]/64`、`spectrum[2b+1] = conj(F[63-b])/64`; @@ -47,25 +42,13 @@ flush 容量;分析与合成两级各自持有状态,互不共享。其它 band 1 与 band 3 取共轭;高频带原样延迟 3 slot; - QMF 合成保留 9 slot 历史,用 10 个 lag 的 prototype 匹配累加,奇数 lag 取虚部。 -`data/rosella_captured_kernels.npz` 保存捕获版的 prototype(`float32[64,10]`)、 +`data/rosella_captured_kernels.npz` 保存 11-tap 滤波器组的 prototype(`float32[64,10]`)、 rotation(`complex64[64]`)与 hybrid 系数(`complex64[22]`);载入后提升为 -float64/complex128。这些数值与 C++ 内核编译进去的 -`src/hrtf/rosella_filterbank_tables.h` 相同,不包含个性化 HRTF 或 room 调参。 +float64/complex128,不包含个性化 HRTF 或 room 调参。 运行时的起始延迟跳过与 flush 容量都取自所选滤波器组:`961` 或 `448` samples, `.report.json` 的 `binaural_renderer_backend` 用 `filterbank`、`filterbank_name` 与 `latency_compensated_samples` 记录实际选择。 -## 验证 - -本地回归脚本放在 `tests/`(该目录不随仓库跟踪,与其余本地验证脚本一致),覆盖: - -- 捕获版四个 half-step 的参考向量(与 C++ 内核 `tests/rosella_filterbank_vectors.h` - 同一组、由独立 NumPy 实现生成),逐样本误差小于 `1e-10`; -- 两套滤波器组的脉冲往返延迟分别落在 `961` 与 `448` samples; -- 分块处理(含 0 长度与不等长切分)与 `reset()` 不变性; -- 两套滤波器组互不干扰:交叉处理一个 bank 不改变另一个 bank 的状态; -- 越界取值被拒绝。 - -`docs/binaural.md` 描述的公开滤波器组公式与 `.jochrtf` 方向和公开版一致;捕获版只 -作用于 Rosella 路径。 +`docs/binaural.md` 描述的公开滤波器组公式与 `.jochrtf` 方向和公开版一致;11-tap 版本 +只作用于 Rosella 路径。 diff --git a/src/adm_atmos.py b/src/adm_atmos.py index 85021c0..5c9886f 100644 --- a/src/adm_atmos.py +++ b/src/adm_atmos.py @@ -209,6 +209,18 @@ def _balance_byte(value): return magnitude | (0x80 if value < 0 else 0) +def warp_mode(atmos_meta): + """OAMD trim element 的 ``warp_mode`` → 下混模式码。 + + dbmd 的 warp 字段、ADM 的 ``DolbyWarpMode`` 和 DAMF 的 ``WarpMode`` 共用 + 一套编号:0 normal、1 warping、2 downmix_pl2x、3 downmix_loro,4 表示未 + 指定。来源只有 2 bit 的 OAMD 字段,因此取值落在 0..3。 + """ + if not atmos_meta: + return 0 + return max(0, min(3, int(atmos_meta.get("warp_mode", 0)))) + + def build_dbmd(object_count=25, joc_binaural_mode=4, atmos_meta=None): """segment 10:9 个 trim 配置 + 逐轨 bypass + JOC object slots 10..24 的 mode。 @@ -233,6 +245,9 @@ def build_dbmd(object_count=25, joc_binaural_mode=4, atmos_meta=None): at[103] = 0x03 at[106] = 0x01 at[111] = 0x22; at[112] = 0xFF + # at[152] 高 2 位是 bed_distribution、低 3 位是 warp;这里只写 warp, + # 其余位保持模板原值。 + at[152] = (at[152] & 0xF8) | warp_mode(atmos_meta) out.append(9); out += struct.pack(" dict[str, np.ndarray]: def load_kernel_tables(path: str | Path = DEFAULT_CAPTURED_KERNEL_DATA ) -> dict[str, np.ndarray]: - """Load and cache the captured Rosella kernel tables.""" + """Load and cache the 11-tap Rosella bank's kernel tables.""" return _load_tables(str(Path(path).expanduser().resolve())) diff --git a/src/speaker_backend.py b/src/speaker_backend.py index 729d448..8f2efa4 100644 --- a/src/speaker_backend.py +++ b/src/speaker_backend.py @@ -12,15 +12,17 @@ FRAME_SAMPLES = 1536 def create_speaker_renderer(layout: str | SpeakerLayout, bed_channels=None, - trim_meta=None): + trim_meta=None, *, parameters=None): """Create a stateful speaker renderer and return ``(renderer, info)``.""" target = get_speaker_layout(layout) if isinstance(layout, str) else layout renderer = PythonSpeakerRenderer(target, bed_channels=bed_channels, - trim_meta=trim_meta) + trim_meta=trim_meta, parameters=parameters) return renderer, { "name": "python", "layout": target.name, "bed_objects": len(getattr(renderer, "bed_targets", {}) or {}), + "initial_warp_mode": renderer.parameters.warp_mode, + "initial_trim_mode": renderer.parameters.trim_mode, } diff --git a/src/speaker_renderer.py b/src/speaker_renderer.py index 517245a..72153b5 100644 --- a/src/speaker_renderer.py +++ b/src/speaker_renderer.py @@ -12,9 +12,10 @@ from typing import Iterable import numpy as np -from oamd_bits import JocFieldState, frame_update +from oamd_bits import JocFieldState, frame_update, parse_trim from speaker_layouts import RegionGeometry, SpeakerLayout, get_speaker_layout -from trim_balance import apply_balance, balance_layers, channel_gains +from trim_balance import (SpeakerRenderParameters, apply_balance, + explicit_position_gain, parameters_from_oamd) Q15_SCALE = 32768.0 Q15_MAX = 32767.0 / Q15_SCALE @@ -35,9 +36,9 @@ def bed_object_targets(bed_channels) -> dict[int, tuple[float, float, float]]: def _all_object_targets(layout: SpeakerLayout, state: JocFieldState, - bed_targets=None, balance=(0.0, 0.0)) -> np.ndarray: + bed_targets=None, parameters=None) -> np.ndarray: bed_targets = bed_targets or {} - listener, overhead = balance + parameters = parameters or parameters_from_oamd(None, layout) result = np.zeros((15, layout.channel_count), dtype=np.float64) for object_index in range(1, 16): fixed = bed_targets.get(object_index) @@ -47,9 +48,9 @@ def _all_object_targets(layout: SpeakerLayout, state: JocFieldState, u = state.q[(object_index, "q1")] / Q15_SCALE v = state.q[(object_index, "q2")] / Q15_SCALE w = state.q[(object_index, "q3")] / Q15_SCALE - v = apply_balance(v, w, listener, overhead) result[object_index - 1] = render_point_gains( - layout, u, v, w, standard_order=False) + layout, u, v, w, standard_order=False, parameters=parameters, + bypass_trim=parameters.bypasses(object_index)) return result @@ -175,16 +176,30 @@ def _plane_gains(region: RegionGeometry, coordinates: np.ndarray, groups, def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float, *, region_index: int = 0, enable_height: bool = True, - object_gain: float = 1.0, standard_order: bool = True) -> np.ndarray: + object_gain: float = 1.0, standard_order: bool = True, + parameters: SpeakerRenderParameters | None = None, + bypass_trim: bool = False) -> np.ndarray: """Render one point object to a target speaker layout using float64.""" target = get_speaker_layout(layout) if isinstance(layout, str) else layout + controls = parameters or parameters_from_oamd(None, target) region = target.regions[int(region_index)] coordinates = _coordinates(region) - floor_v = min(max(math.ldexp(float(v), floor_y_exponent(target.speaker_bitfield)), 0.0), 1.0) + v, w = float(v), max(float(w), 0.0) + floor_v = min(max(math.ldexp(v, floor_y_exponent( + target.speaker_bitfield, controls.warp_mode == 1)), 0.0), 1.0) + top_v = min(max(v, 0.0), 1.0) + trim_v = v + if controls.trim_mode == "explicit" and not bypass_trim: + # Apply warp -> balance for floor, balance(original Y) for + # top and trim. These coordinates must not be collapsed into one Y. + listener, overhead = controls.listener_balance, controls.overhead_balance + floor_v = apply_balance(floor_v, w, listener, overhead) + top_v = apply_balance(top_v, w, listener, overhead) + trim_v = apply_balance(v, w, listener, overhead) floor = _plane_gains(region, coordinates, region.axis0_groups, u, floor_v, region.mode) point_gains = floor if region.mode == 3: - top = _plane_gains(region, coordinates, region.axis1_groups, u, float(v), 3) + top = _plane_gains(region, coordinates, region.axis1_groups, u, top_v, 3) height = min(max(float(w) if enable_height else 0.0, 0.0), Q15_MAX) if height >= Q15_MAX: point_gains = top @@ -192,7 +207,12 @@ def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float floor_weight, height_weight = equal_power_pair(height) point_gains = floor * floor_weight + top * height_weight internal = np.zeros(target.channel_count, dtype=np.float64) - gain = position_gain(target.speaker_bitfield, v, w) * float(object_gain) + gain = float(object_gain) + if not bypass_trim: + if controls.trim_mode == "automatic": + gain *= position_gain(target.speaker_bitfield, trim_v, w) + elif controls.trim_mode == "explicit": + gain *= explicit_position_gain(controls, u, trim_v, w) for point_gain, speaker_id in zip(point_gains, region.speaker_ids): internal[speaker_id] = point_gain * gain if standard_order: @@ -213,18 +233,20 @@ class PythonSpeakerRenderer: """Stateful float64 speaker renderer.""" def __init__(self, layout: str | SpeakerLayout, block_size: int = 32, - bed_channels=None, trim_meta=None): + bed_channels=None, trim_meta=None, *, parameters=None): self.layout = get_speaker_layout(layout) if isinstance(layout, str) else layout self.block_size = int(block_size) if self.block_size <= 0: raise ValueError("block_size must be positive") self.bed_targets = bed_object_targets(bed_channels) - self.trim_gains = np.asarray(channel_gains(trim_meta, self.layout), - dtype=np.float64) - self.balance = balance_layers(trim_meta, self.layout) + if parameters is not None and trim_meta is not None: + raise ValueError("pass normalized parameters or OAMD trim_meta, not both") + self._parameters_override = parameters + self._initial_parameters = parameters or parameters_from_oamd(trim_meta, self.layout) self.reset() def reset(self): + self.parameters = self._initial_parameters channels = self.layout.channel_count self._state = JocFieldState() self._current = np.zeros((15, channels), dtype=np.float64) @@ -235,9 +257,13 @@ class PythonSpeakerRenderer: def _apply_payload(self, payload): update = frame_update(payload) + if self._parameters_override is None: + trim = parse_trim(payload) + if trim is not None: + self.parameters = parameters_from_oamd(trim, self.layout) self._state.apply(update["values"]) new_target = _all_object_targets(self.layout, self._state, self.bed_targets, - self.balance) + self.parameters) blocks = ramp_block_count(update["ramp_duration_samples"], self.block_size) difference = new_target - self._current significant = np.abs(difference) >= GAIN_SNAP_THRESHOLD @@ -306,7 +332,6 @@ class PythonSpeakerRenderer: for payload in events_by_block.get(total_blocks, ()): self._apply_payload(payload) - output *= self.trim_gains[None, :] if standard_order: return output[:, list(self.layout.internal_to_standard)] return output @@ -321,9 +346,11 @@ class PythonSpeakerRenderer: def render_objects16(objects16: np.ndarray, payload_events: Iterable[tuple[int, bytes]], layout: str | SpeakerLayout, *, block_size: int = 32, - standard_order: bool = True, output: np.ndarray | None = None) -> np.ndarray: + standard_order: bool = True, output: np.ndarray | None = None, + trim_meta=None, parameters=None) -> np.ndarray: """Render a complete interleaved LFE + 15 object stream.""" - renderer = PythonSpeakerRenderer(layout, block_size=block_size) + renderer = PythonSpeakerRenderer(layout, block_size=block_size, + trim_meta=trim_meta, parameters=parameters) return renderer.process(objects16, payload_events, standard_order=standard_order, output=output) def payload_events_from_index(index, frame_count: int, *, frame_samples: int = 1536, diff --git a/src/trim_balance.py b/src/trim_balance.py index ade01a4..9c435de 100644 --- a/src/trim_balance.py +++ b/src/trim_balance.py @@ -24,6 +24,11 @@ the axis and it leaves the two walls untouched. See ``front_back``. """ from __future__ import annotations +from dataclasses import dataclass +import math + +from variant_error import UnsupportedVariantError + # Layouts that carry trim and front-back balance semantics, by their compact # names: 2.0, 3.1, 4.0, 5.1, 5.1.2, 5.1.4 and 7.1. Wider layouts leave the # programme untrimmed. @@ -49,6 +54,93 @@ BALANCE_STEPS = 16 WALL = 0.05 +@dataclass(frozen=True) +class SpeakerRenderParameters: + """Codec-independent, per-layout speaker controls; balances are in [-1,1]. + + Warp 2/3 intentionally have no rendering implementation yet. An absent + warp indication uses normal (0); no CLI override is provided. + """ + warp_mode: int = 0 + trim_mode: str = "automatic" + surround_db: float = 0.0 + height_db: float = 0.0 + centre_db: float = 0.0 + listener_balance: float = 0.0 + overhead_balance: float = 0.0 + disable_per_object: tuple[bool, ...] = () + + def __post_init__(self): + if self.warp_mode in (2, 3): + raise UnsupportedVariantError( + "speaker", "warp_mode_not_implemented", + "Speaker rendering implements warp modes 0/1 only; 2/3 are reserved in E-AC-3 JOC", + details={"warp_mode": self.warp_mode}) + if self.warp_mode not in (0, 1): + raise ValueError(f"invalid warp mode: {self.warp_mode!r}") + if self.trim_mode not in ("automatic", "disabled", "explicit"): + raise ValueError(f"invalid trim mode: {self.trim_mode!r}") + for value in (self.surround_db, self.height_db, self.centre_db): + if not math.isfinite(value): + raise ValueError("trim levels must be finite") + for value in (self.listener_balance, self.overhead_balance): + if not math.isfinite(value) or not -1.0 <= value <= 1.0: + raise ValueError("balance must be finite and in [-1,1]") + + def bypasses(self, object_index): + return (0 <= object_index < len(self.disable_per_object) + and self.disable_per_object[object_index]) + + +def parameters_from_oamd(atmos_meta, layout): + """Adapt OAMD calibration to one layout, without changing source positions. + + A config's default/disabled flag overrides global custom mode. Missing + custom configuration falls back to automatic. + """ + meta = atmos_meta or {} + common = dict(warp_mode=meta.get("warp_mode", 0), + disable_per_object=tuple(meta.get("disable_per_obj") or ())) + if not _supported(layout): + return SpeakerRenderParameters(**common, trim_mode="disabled") + mode = meta.get("global_trim_mode", 0) + if mode == 1: + return SpeakerRenderParameters(**common, trim_mode="disabled") + if mode not in (0, 1, 2): + raise UnsupportedVariantError( + "speaker", "trim_mode_not_implemented", "Reserved OAMD global trim mode", + details={"global_trim_mode": mode}) + record = _record(meta, layout) if mode == 2 else None + if not isinstance(record, dict) or record.get("default"): + return SpeakerRenderParameters(**common) + if record.get("disabled"): + return SpeakerRenderParameters(**common, trim_mode="disabled") + # Surround/height are attenuation-only. Centre retains its signed level. + return SpeakerRenderParameters( + **common, trim_mode="explicit", + surround_db=min(float(record.get("surround", 0.0)), 0.0), + height_db=min(float(record.get("height", 0.0)), 0.0), + centre_db=float(record.get("centre", 0.0)), + listener_balance=float(record.get("bal3d_y_lis", 0.0)), + overhead_balance=float(record.get("bal3d_y_tb", 0.0))) + + +def explicit_position_gain(parameters, x, y, z): + """Position-dependent *object* trim, evaluated at balance(original Y). + + Surround fades in over Y=0..0.5 and out towards Z=1; height fades in + with Z. Centre trim is local to the front-centre region. The resulting + scalar multiplies the object's entire speaker gain vector, before mixing. + """ + surround = min(max(2.0 * y, 0.0), 1.0) * (1.0 - z) + centre = 1.0 - min(max(10.0 * max( + abs(z) - WALL, y - WALL, abs(x - 0.5) - WALL), 0.0), 1.0) + db = (surround * min(parameters.surround_db, 0.0) + + z * min(parameters.height_db, 0.0) + + centre * parameters.centre_db) + return 10.0 ** (db / 20.0) + + def _height_class(count): if count == 0: return 0 @@ -106,7 +198,10 @@ def group_levels_db(atmos_meta, layout): def channel_gains(atmos_meta, layout): - """Per-channel linear gain for the layout, in channel order.""" + """Legacy diagnostic group levels, not the speaker rendering operation. + + Rendering uses explicit_position_gain per object before accumulation. + """ levels = group_levels_db(atmos_meta, layout) gains = [] for label in getattr(layout, "channels", ()):