Render SOFA HRIRs directly and place bed channels by their labels
Two changes that belong to the same reproduction model: a bed channel has one position, defined by its label, and a SOFA set keeps the impulse responses it was measured with. CBA bed channels: the position comes from the channel's label in ETSI TS 103 420 Table B.10, not from the layout the decoder is asked to render to. The per-layout anchor table displaced bed channels onto the nearest target speaker, which silently moved content whenever the target layout changed. - Drop the anchor table and the geometry lookup; bed_channel_q15 is the only source of bed positions, so stream and task no longer need a layout to ask for one - Keep the 17-bit non-standard bed mask in channel order, as Table 13 / TS 103 190-2 Table 65 transmit it - Exclude LFE channels from the object list while keeping them in the coverage mask - Cover the Table B.10 coordinates, the q15 encoding, the 10-bit and 17-bit masks and the LFE split in tests SOFA rendering: compiling a SOFA set into a fifth-order directional field threw away the measured impulse response - the SH projection, the 64-QMF/77-hybrid round trip and the synthetic room replaced it. The full IRs are now kept and rendered as such, with the reproduction defined by a configuration that names a personalized model rather than by the renderer itself. - Add SphericalMesh: incremental convex hull over the measurement directions, barycentric weights inside the containing triangle and spherical Voronoi areas for the centre spectrum - Add FullHrirField: rfft of the selected shell, 200-1500 Hz phase fit for the delay, delay-aligned spectra, log-magnitude/phase or MCA interpolation, rigid-sphere radial ratio, recording origin and the common FFT/2 latency - Add Uniform/MultiSourcePartitionedConvolver: block 128, numpy FFT conventions, zero-fade filter rebuild from the blocks in flight - Move the Rosella reference arithmetic (profile geometry, direction basis, direct gains and room send, table-A room) out of the renderer into rosella_reference so both renderers share it - Add ReferenceEnvironment and SofaReferenceRuntime: JSON configuration with model hash and distance-profile mapping, level calibration over equal-area directions, the reference room, 1536-sample batching and a single latency removal - Keep .jochrtf an explicit input and the SOFA compile step behind --hrtf-cache-policy/--hrtf-cache-dir; add --reference-config and --sofa-interpolation, and make --hrtf-radius-m optional for a single shell - Cover the mesh weights and Voronoi areas, the sphere series and the partitioned convolver with reference values in tests - Document both rules in the READMEs and the binaural notes Rosella renders stay byte-identical (d5877e89...) and so do renders of the legacy compiled path (6f0a75d2...). Direct SOFA renders match the Python reference to 1e-4 in level; the residual comes from the barycentric solve (analytic inverse here, LAPACK there) on near-degenerate triangles.
This commit is contained in:
1 parent
34a722f56a
commit
fd842db26f
27 files changed
+4746
-1065
No files matched your search
+69
-17
@@ -5,28 +5,70 @@
|
||||
JustOneCacophony's binaural backend supports three HRTF sources:
|
||||
`SimpleFreeFieldHRIR` SOFA, the Rosella `.personalized_headphone` model exported
|
||||
by Dolby's official personalization scan (its JSON parsing is implemented by
|
||||
this project and invokes no Dolby software), and the `.jochrtf` cache compiled
|
||||
from SOFA. SOFA is compiled into an in-memory directional field when the model
|
||||
is loaded. A `.jochrtf` file is only a disposable, reproducible JOC compiled
|
||||
HRTF cache; it is neither an interchange format nor a prerequisite for using
|
||||
SOFA.
|
||||
this project and invokes no Dolby software), and `.jochrtf` (a JOC compiled HRTF
|
||||
cache).
|
||||
|
||||
SOFA keeps its full HRIRs and renders directly through a configured reference
|
||||
environment; it is no longer compiled into a directional field:
|
||||
|
||||
```text
|
||||
SOFA FIR
|
||||
-> CanonicalHrtf
|
||||
-> 48 kHz / one radius shell / delay-phase policy
|
||||
-> 64-QMF / 77-hybrid projection
|
||||
-> fifth-order ACN/N3D real-SH field
|
||||
-> per-object direct + early reflections
|
||||
-> shared unitary-FDN late room
|
||||
-> float64 stereo
|
||||
SOFA full HRIR [M,2,N]
|
||||
-> CanonicalHrtf (48 kHz / one radius shell)
|
||||
-> delay-aligned complex spectra (200-1500 Hz phase fit, Data.Delay included)
|
||||
-> spherical-triangle barycentric interpolation (lf_logmag_phase or lf_mca_zero)
|
||||
-> rigid-sphere radial correction (reference-environment distance)
|
||||
-> per-object partitioned convolution (block 128, FFT = max(2048, >=2*support))
|
||||
-> reference-environment recursive room model
|
||||
-> float64 stereo (one common latency removed)
|
||||
```
|
||||
|
||||
`.jochrtf` stays an explicit input (`--compiled-hrtf-cache`) and keeps the
|
||||
real-SH field renderer; passing `--hrtf-cache-policy` / `--hrtf-cache-dir`
|
||||
selects the legacy "SOFA -> `.jochrtf`" compile path.
|
||||
|
||||
## Reference environment
|
||||
|
||||
Direct SOFA rendering needs a reference-environment configuration
|
||||
(`--reference-config`, default `<exe>/local/binaural_reference.json`). It names
|
||||
the Rosella `.personalized_headphone` model together with its SHA-256, the
|
||||
`profile_index` used as the level anchor (2 = far, 3 = mid), the room impulse
|
||||
slots (4096 by default) and the number of level-calibration directions (128).
|
||||
|
||||
Every SOFA direction is first mapped into the reference reproduction through the
|
||||
model's distance profile:
|
||||
|
||||
```text
|
||||
ADM position -> int16 metadata grid -> distance profile
|
||||
-> unit direction + distance scale
|
||||
-> direct gain a = 1/sqrt(1+coupling*(h0*d)^2) (1 for near)
|
||||
-> room send = h0*d*a (0 for near)
|
||||
-> near-centre regularization: ear distance = max(d, minimum_distance)
|
||||
```
|
||||
|
||||
`near|mid|far` (`--binaural-mode`) selects one of the model's three distance
|
||||
profiles; level calibration always uses the configured `profile_index` as its
|
||||
anchor, so all three modes share one level.
|
||||
|
||||
The direct path is the interpolated, delay-aligned full HRIR; the room send only
|
||||
feeds the reference room model (the LFE channel never does). Their latencies come
|
||||
from the larger of the field (FFT/2 plus the recording origin) and the bank
|
||||
(961), and are removed once before the output.
|
||||
|
||||
## Inputs
|
||||
|
||||
The binaural backend consumes a compiled directional field (a JOC compiled HRTF cache,
|
||||
`.jochrtf`) plus the shared filterbank tables. Both are read-only inputs: this library
|
||||
performs no parsing or conversion of measurement data formats.
|
||||
A SOFA input renders directly from its full HRIRs and only needs the SOFA file and
|
||||
the reference-environment configuration:
|
||||
|
||||
```powershell
|
||||
joc_cli input.m4a --binaural `
|
||||
--sofa-hrtf HRTF\subject.sofa `
|
||||
--reference-config local\binaural_reference.json `
|
||||
--binaural-mode mid --sofa-interpolation lf_logmag_phase
|
||||
```
|
||||
|
||||
A `.jochrtf` input (and `.personalized_headphone`) also needs the shared filterbank
|
||||
tables. Both are read-only inputs; parsing and coordinate validation happen while the
|
||||
model is loaded, and nothing is converted again during rendering.
|
||||
|
||||
```powershell
|
||||
joc_cli input.m4a --binaural `
|
||||
@@ -35,8 +77,14 @@ joc_cli input.m4a --binaural `
|
||||
--binaural-mode mid --binaural-tail-seconds 5.0
|
||||
```
|
||||
|
||||
A multi-radius SOFA requires `--hrtf-radius-m`; a single shell is selected
|
||||
automatically. `--sofa-interpolation` picks the local interpolation of the full HRIR:
|
||||
`lf_logmag_phase` (default, log magnitude plus phase) or `lf_mca_zero` (auditory-band
|
||||
magnitude correction).
|
||||
|
||||
The library exposes the same fields: `joc_task_config` for a file task,
|
||||
`joc_stream_config` for streaming, where `hrtf_path`, `kernels_path`,
|
||||
`joc_stream_config` for streaming, where `hrtf_sofa_path` (or `hrtf_path`),
|
||||
`reference_config_path`, `sofa_interpolation`, `hrtf_compile`, `kernels_path`,
|
||||
`binaural_mode` and `binaural_tail_seconds` configure the binaural path.
|
||||
|
||||
The `.jochrtf` file is an **input**, not a product of this library: compiling it from
|
||||
@@ -110,6 +158,10 @@ The compiler recognizes three mutually exclusive representations:
|
||||
|
||||
No path may add a second ear delay or phase-group delay.
|
||||
|
||||
The direct SOFA path does not use this compile-time policy: it keeps the full IRs and
|
||||
estimates the arrival time with a 200-1500 Hz phase fit, so `Data.Delay` and the fitted
|
||||
delay are each applied once (the delay is removed first and restored afterwards).
|
||||
|
||||
## Public filterbank and directional field
|
||||
|
||||
The SOFA and `.jochrtf` runtime is fixed at:
|
||||
|
||||
+57
-14
@@ -4,25 +4,60 @@
|
||||
|
||||
JustOneCacophony 的双耳后端支持三种 HRTF 来源:`SimpleFreeFieldHRIR` SOFA、
|
||||
杜比官方软件个性化扫描导出的 Rosella `.personalized_headphone`(JSON 解析由本项目
|
||||
自行实现,不调用杜比软件),以及从 SOFA 编译出的 `.jochrtf` 缓存。SOFA 在模型加载
|
||||
时编译成内存方向场;`.jochrtf` 只是可删除、可重建的 JOC compiled HRTF cache,
|
||||
不是交换格式,也不是使用 SOFA 的前置步骤。
|
||||
自行实现,不调用杜比软件),以及 `.jochrtf`(JOC compiled HRTF cache)。
|
||||
|
||||
SOFA 默认保留完整 HRIR,按“参考环境”直连渲染,不再编译成方向场:
|
||||
|
||||
```text
|
||||
SOFA FIR
|
||||
-> CanonicalHrtf
|
||||
-> 48 kHz / 单 radius shell / delay-phase policy
|
||||
-> 64-QMF / 77-hybrid projection
|
||||
-> 五阶 ACN/N3D 实球谐场
|
||||
-> 逐对象 direct + early reflections
|
||||
-> shared unitary-FDN late room
|
||||
-> stereo float64
|
||||
SOFA 完整 HRIR [M,2,N]
|
||||
-> CanonicalHrtf(48 kHz / 单 radius shell)
|
||||
-> 延迟对齐的复数频谱(200-1500 Hz 相位拟合,含 Data.Delay)
|
||||
-> 球面三角形重心插值(lf_logmag_phase 或 lf_mca_zero)
|
||||
-> 刚性球径向修正(按参考环境距离)
|
||||
-> 逐对象 partitioned convolution(block 128,FFT = max(2048, ≥2×support))
|
||||
-> 参考环境的递归房间模型
|
||||
-> stereo float64(统一扣除 latency)
|
||||
```
|
||||
|
||||
`.jochrtf` 仍是显式输入(`--compiled-hrtf-cache`),走球谐方向场渲染;显式给出
|
||||
`--hrtf-cache-policy` / `--hrtf-cache-dir` 会回到旧的“SOFA → `.jochrtf`”编译路径。
|
||||
|
||||
## 参考环境
|
||||
|
||||
SOFA 直连渲染需要一个参考环境配置(`--reference-config`,默认
|
||||
`<exe>/local/binaural_reference.json`)。配置给出 Rosella `.personalized_headphone`
|
||||
模型的路径与 SHA-256、`profile_index`(2=far、3=mid,作为 level 校准锚点)、
|
||||
`room_impulse_slots`(默认 4096)与 `level_calibration_directions`(默认 128)。
|
||||
|
||||
渲染时每个 SOFA 方向先经该模型的 distance profile 映射成参考环境中的位置:
|
||||
|
||||
```text
|
||||
ADM position -> int16 metadata grid -> distance profile
|
||||
-> direction(单位向量)+ 距离 scale
|
||||
-> direct gain a = 1/sqrt(1+coupling·(h0·d)^2)(near 恒为 1)
|
||||
-> room send = h0·d·a(near 恒为 0)
|
||||
-> 近中心正则化:ear distance 取 max(d, minimum_distance)
|
||||
```
|
||||
|
||||
`near|mid|far` 选择该模型的三个 distance profile 之一(`--binaural-mode`),level 校准
|
||||
始终用配置里的 `profile_index` 作为锚点,因此三种模式共享同一个 level。
|
||||
|
||||
指向同一声道的直达声是延迟对齐后的完整 HRIR 插值结果,房间 send 只进参考环境的
|
||||
房间模型(LFE 声道不进房间)。两者的 latency 由 field(FFT/2 与 recording origin)
|
||||
和 bank(961)中的较大者决定,输出前统一扣除。
|
||||
|
||||
## 输入接口
|
||||
|
||||
双耳后端消费一个已编译的方向场(JOC compiled HRTF cache,`.jochrtf`)与共享滤波器组表;
|
||||
两者都是只读输入,本库不做任何测量数据格式的解析或转换:
|
||||
SOFA 输入直接按完整 HRIR 渲染,只需要 SOFA 与参考环境配置:
|
||||
|
||||
```powershell
|
||||
joc_cli input.m4a --binaural `
|
||||
--sofa-hrtf HRTF\subject.sofa `
|
||||
--reference-config local\binaural_reference.json `
|
||||
--binaural-mode mid --sofa-interpolation lf_logmag_phase
|
||||
```
|
||||
|
||||
`.jochrtf` 输入(以及 `.personalized_headphone`)需要共享滤波器组表,两者都是只读输入:
|
||||
|
||||
```powershell
|
||||
joc_cli input.m4a --binaural `
|
||||
@@ -31,8 +66,13 @@ joc_cli input.m4a --binaural `
|
||||
--binaural-mode mid --binaural-tail-seconds 5.0
|
||||
```
|
||||
|
||||
多球壳 SOFA 必须用 `--hrtf-radius-m` 指定球壳(单球壳自动选择);`--hrtf-radius-m`
|
||||
对已编译的 `.jochrtf` 不再生效。`--sofa-interpolation` 选择完整 HRIR 的局部插值:
|
||||
`lf_logmag_phase`(默认,对数幅度 + 相位)或 `lf_mca_zero`(听觉带幅度修正)。
|
||||
|
||||
库接口使用同一组字段:文件任务用 `joc_task_config`,流式用 `joc_stream_config`,
|
||||
其中 `hrtf_path`、`kernels_path`、`binaural_mode`、`binaural_tail_seconds` 决定双耳通路。
|
||||
其中 `hrtf_sofa_path`(或 `hrtf_path`)、`reference_config_path`、`sofa_interpolation`、
|
||||
`hrtf_compile`、`kernels_path`、`binaural_mode`、`binaural_tail_seconds` 决定双耳通路。
|
||||
|
||||
`.jochrtf` 是**输入**而不是本库的产物:从 SOFA 或测量数据编译该缓存属于工具链的职责,
|
||||
与本仓库解耦。缓存加载时会校验成员集合、dtype 与形状、C 连续性、CRC-32 以及按成员重算的
|
||||
@@ -96,6 +136,9 @@ convention adapter,不能只通过 reshape 进入核心 importer。
|
||||
|
||||
任何路径都不能再叠加第二套 ear delay 或 phase-group delay。
|
||||
|
||||
SOFA 直连路径不走这套编译期语义:它保留完整 IR,用 200-1500 Hz 相位拟合估计到达时间,
|
||||
`Data.Delay` 与拟合出的延迟各应用一次(先去掉 `Data.Delay`,再用插值后的延迟复原)。
|
||||
|
||||
## 公开 filterbank 与方向场
|
||||
|
||||
运行时固定为:
|
||||
|
||||
Reference in new issue
Block a user