Add the captured Rosella filterbank and select it per bank
The Rosella path now chooses its filterbank. Bank 1 is the public bank
the reference implementation has always used (13 taps / 6 slots,
961 samples); bank 2 is the captured bank the reference headphone
renderer actually runs (11 taps / 3 slots, 448 samples).
- src/rosella_captured_filterbank.py implements the four half-steps in
float64/complex128; the captured prototype, rotation and hybrid
coefficients live in data/rosella_captured_kernels.npz
- src/rosella_bank.py owns the choice: both banks expose the same four
stages, each with its own coefficients, history and latency
- RosellaBinauralRenderer takes filterbank= and derives the startup
skip, the flush capacity and the report fields from it
- --rosella-filterbank 1|2 selects the bank, default 1; it is rejected
for the SOFA and .jochrtf inputs instead of being ignored
- document both banks in docs/rosella_filterbank.{md,en.md}, and point
the binaural docs, the data README and the options tables at it
Checked: the captured stages reproduce the reference vectors the C++
core checks (within 1e-10), an impulse returns at 961 and 448 samples,
and chunking, reset and bank independence hold for both.
This commit is contained in:
1 parent
247e2a90b6
commit
0e98e91762
13 files changed
+606
-24
No files matched your search
+19
-2
@@ -22,7 +22,7 @@ This is research code, not a complete, standards-compliant, or production-grade
|
|||||||
or `9.1.6` (`4.0` is the quad layout, a front and a back pair; see
|
or `9.1.6` (`4.0` is the quad layout, a front and a back pair; see
|
||||||
[Speaker layouts and channel order](#speaker-layouts-and-channel-order)).
|
[Speaker layouts and channel order](#speaker-layouts-and-channel-order)).
|
||||||
- Run public SOFA binaural rendering directly from `pcm16 + ID11/OAMD`, without a temporary ADM BWF.
|
- Run public SOFA binaural rendering directly from `pcm16 + ID11/OAMD`, without a temporary ADM BWF.
|
||||||
- Keep the binaural DSP in float64/complex128, including 961-sample latency compensation, cross-frame state, and the room tail.
|
- Keep the binaural DSP in float64/complex128, including the selected filterbank's latency compensation (961 or 448 samples), cross-frame state, and the room tail.
|
||||||
- Use a shared float32/PCM24 WAV writer and explicit PCM24 clipping policy for direct outputs.
|
- Use a shared float32/PCM24 WAV writer and explicit PCM24 clipping policy for direct outputs.
|
||||||
- Implement everything in pure Python on NumPy;
|
- Implement everything in pure Python on NumPy;
|
||||||
- Read or write metadata sidecars and produce metadata, timing, and output reports.
|
- Read or write metadata sidecars and produce metadata, timing, and output reports.
|
||||||
@@ -105,6 +105,8 @@ python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
|||||||
--binaural-mode near
|
--binaural-mode near
|
||||||
python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
||||||
--hrtf-cache-policy disk
|
--hrtf-cache-policy disk
|
||||||
|
python main.py input.m4a --binaural --personalized-headphone `
|
||||||
|
--rosella-filterbank 2
|
||||||
python main.py input.m4a --binaural --binaural-output output.binaural.wav
|
python main.py input.m4a --binaural --binaural-output output.binaural.wav
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -223,7 +225,21 @@ sample.
|
|||||||
|
|
||||||
### Binaural calculation
|
### Binaural calculation
|
||||||
|
|
||||||
See [Binaural Rendering Mathematics](docs/binaural.en.md) for QMF, hybrid processing, direction fields, distance, ITD, room processing, the 512-sample parameter updates above, and 961-sample latency compensation.
|
See [Binaural Rendering Mathematics](docs/binaural.en.md) for QMF, hybrid processing, direction fields, distance, ITD, room processing, the 512-sample parameter updates above, and the selected filterbank's latency compensation (961 by default, 448 with `--rosella-filterbank 2`).
|
||||||
|
|
||||||
|
### Rosella filterbank
|
||||||
|
|
||||||
|
`--rosella-filterbank 1|2` selects the filterbank of the `.personalized_headphone`
|
||||||
|
path:
|
||||||
|
|
||||||
|
- `1` (default): the public bank, 13-tap hybrid / 6-slot high-band delay,
|
||||||
|
961 samples of latency;
|
||||||
|
- `2`: the Rosella bank captured from the reference renderer, 11-tap hybrid /
|
||||||
|
3-slot high-band delay, 448 samples of latency.
|
||||||
|
|
||||||
|
The option only applies to the Rosella model; the SOFA and `.jochrtf` paths
|
||||||
|
reject it. See [Rosella filterbank](docs/rosella_filterbank.en.md) for the
|
||||||
|
parameters, buffers, latency and validation of both banks.
|
||||||
|
|
||||||
For all options:
|
For all options:
|
||||||
|
|
||||||
@@ -321,3 +337,4 @@ See the [mathematical notes](docs/math.en.md) for the equations used by the deco
|
|||||||
|
|
||||||
- [Mathematical notes](docs/math.en.md) · [中文](docs/math.md)
|
- [Mathematical notes](docs/math.en.md) · [中文](docs/math.md)
|
||||||
- [Binaural rendering](docs/binaural.en.md) · [中文](docs/binaural.md)
|
- [Binaural rendering](docs/binaural.en.md) · [中文](docs/binaural.md)
|
||||||
|
- [Rosella filterbank](docs/rosella_filterbank.en.md) · [中文](docs/rosella_filterbank.md)
|
||||||
@@ -20,7 +20,7 @@
|
|||||||
- 直接渲染 `2.0`、`3.1`、`4.0`、`5.1`、`7.1`、`5.1.2`、`5.1.4`、`7.1.2`、`7.1.4`、`9.1.4`、
|
- 直接渲染 `2.0`、`3.1`、`4.0`、`5.1`、`7.1`、`5.1.2`、`5.1.4`、`7.1.2`、`7.1.4`、`9.1.4`、
|
||||||
`9.1.6`(`4.0` 是 Quad 布局,前后各一对;声道排列见[扬声器布局与声道顺序](#扬声器布局与声道顺序));
|
`9.1.6`(`4.0` 是 Quad 布局,前后各一对;声道排列见[扬声器布局与声道顺序](#扬声器布局与声道顺序));
|
||||||
- 从 `pcm16 + ID11/OAMD` 直接运行公开 SOFA 双耳渲染,不生成临时 ADM BWF;
|
- 从 `pcm16 + ID11/OAMD` 直接运行公开 SOFA 双耳渲染,不生成临时 ADM BWF;
|
||||||
- 双耳 DSP 全程使用 float64/complex128,并保留 961-sample latency compensation、跨帧状态和 room 尾声;
|
- 双耳 DSP 全程使用 float64/complex128,并保留所选滤波器组的延迟补偿(公开版 961、捕获版 448 samples)、跨帧状态和 room 尾声;
|
||||||
- 直接输出统一支持 float32 或 PCM24 WAV,并在 PCM24 削波前提供明确处理策略;
|
- 直接输出统一支持 float32 或 PCM24 WAV,并在 PCM24 削波前提供明确处理策略;
|
||||||
- 全程使用 NumPy 的纯 Python 实现;
|
- 全程使用 NumPy 的纯 Python 实现;
|
||||||
- 读取或写入 metadata sidecar,并生成元数据、运行时间和输出摘要。
|
- 读取或写入 metadata sidecar,并生成元数据、运行时间和输出摘要。
|
||||||
@@ -103,6 +103,8 @@ python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
|||||||
--binaural-mode near
|
--binaural-mode near
|
||||||
python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
python main.py input.m4a --binaural --sofa-hrtf C:\HRTF\subject.sofa `
|
||||||
--hrtf-cache-policy disk
|
--hrtf-cache-policy disk
|
||||||
|
python main.py input.m4a --binaural --personalized-headphone `
|
||||||
|
--rosella-filterbank 2
|
||||||
python main.py input.m4a --binaural --binaural-output output.binaural.wav
|
python main.py input.m4a --binaural --binaural-output output.binaural.wav
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -192,7 +194,18 @@ outer subpayload offset 与 block offset 落到绝对 sample timeline,再加
|
|||||||
|
|
||||||
### 双耳计算
|
### 双耳计算
|
||||||
|
|
||||||
双耳路径的 QMF、hybrid、方向场、距离、ITD、room、上述 512-sample 参数更新和 961-sample 延迟补偿见[双耳渲染数学](docs/binaural.md)。
|
双耳路径的 QMF、hybrid、方向场、距离、ITD、room、上述 512-sample 参数更新和所选滤波器组的延迟补偿(默认 961,`--rosella-filterbank 2` 为 448 samples)见[双耳渲染数学](docs/binaural.md)。
|
||||||
|
|
||||||
|
### Rosella 滤波器组
|
||||||
|
|
||||||
|
`--rosella-filterbank 1|2` 选择 `.personalized_headphone` 路径使用的滤波器组:
|
||||||
|
|
||||||
|
- `1`(默认):公开滤波器组,13-tap hybrid / 6-slot 高频延迟,961 samples 延迟;
|
||||||
|
- `2`:从参考渲染器捕获的 Rosella 滤波器组,11-tap hybrid / 3-slot 高频延迟,
|
||||||
|
448 samples 延迟。
|
||||||
|
|
||||||
|
该选项只作用于 Rosella 模型;SOFA 与 `.jochrtf` 路径给出时报错。两套实现的参数、
|
||||||
|
缓冲、延迟与验证见 [Rosella 滤波器组](docs/rosella_filterbank.md)。
|
||||||
|
|
||||||
更多参数可查看:
|
更多参数可查看:
|
||||||
|
|
||||||
@@ -283,3 +296,4 @@ JustOneCacophony/
|
|||||||
|
|
||||||
- [数学说明](docs/math.md) · [English](docs/math.en.md)
|
- [数学说明](docs/math.md) · [English](docs/math.en.md)
|
||||||
- [双耳渲染](docs/binaural.md) · [English](docs/binaural.en.md)
|
- [双耳渲染](docs/binaural.md) · [English](docs/binaural.en.md)
|
||||||
|
- [Rosella 滤波器组](docs/rosella_filterbank.md) · [English](docs/rosella_filterbank.en.md)
|
||||||
@@ -72,6 +72,25 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q
|
|||||||
The same values also appear in other public implementations of these standards
|
The same values also appear in other public implementations of these standards
|
||||||
(for example FFmpeg's `aacps_tablegen.h` and `aacsbrdata.h`).
|
(for example FFmpeg's `aacps_tablegen.h` and `aacsbrdata.h`).
|
||||||
|
|
||||||
|
### Captured Rosella filterbank
|
||||||
|
|
||||||
|
`rosella_captured_kernels.npz` holds the bank the reference headphone renderer
|
||||||
|
actually runs (`--rosella-filterbank 2`):
|
||||||
|
|
||||||
|
```text
|
||||||
|
format_version little-endian int32[1]
|
||||||
|
qmf_prototype float32[64,10]
|
||||||
|
rotation complex64[64]
|
||||||
|
hybrid complex64[22]
|
||||||
|
```
|
||||||
|
|
||||||
|
The table values are promoted to float64/complex128 on load. The full
|
||||||
|
analysis/synthesis latency is 448 samples (961 for the public bank). These
|
||||||
|
values come from runtime buffer captures of the reference renderer, not from the
|
||||||
|
public standards above; the C++ core compiles the same data in as
|
||||||
|
`src/hrtf/rosella_filterbank_tables.h`. This file also contains no personalized
|
||||||
|
HRTF or room tuning.
|
||||||
|
|
||||||
Public availability of a standard does not by itself grant permission to
|
Public availability of a standard does not by itself grant permission to
|
||||||
practice related patent claims.
|
practice related patent claims.
|
||||||
|
|
||||||
|
|||||||
@@ -64,6 +64,23 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q
|
|||||||
|
|
||||||
相同数值可在 FFmpeg(`aacps_tablegen.h`、`aacsbrdata.h`)等公开实现中查到。
|
相同数值可在 FFmpeg(`aacps_tablegen.h`、`aacsbrdata.h`)等公开实现中查到。
|
||||||
|
|
||||||
|
### Rosella 捕获版滤波器组
|
||||||
|
|
||||||
|
`rosella_captured_kernels.npz` 保存参考头戴渲染器实际运行的那套滤波器组
|
||||||
|
(`--rosella-filterbank 2`):
|
||||||
|
|
||||||
|
```text
|
||||||
|
format_version little-endian int32[1]
|
||||||
|
qmf_prototype float32[64,10]
|
||||||
|
rotation complex64[64]
|
||||||
|
hybrid complex64[22]
|
||||||
|
```
|
||||||
|
|
||||||
|
表值载入后提升为 float64/complex128;全链 latency 为 448 samples(公开版为 961)。
|
||||||
|
这些数值来自参考渲染器的运行时缓冲捕获,不是上述公开标准的推导值;C++ 内核以
|
||||||
|
`src/hrtf/rosella_filterbank_tables.h` 的形式内置同一组数据。文件同样不含个性化
|
||||||
|
HRTF 或 room 调参。
|
||||||
|
|
||||||
标准可公开获取不等于获准实施相关专利。
|
标准可公开获取不等于获准实施相关专利。
|
||||||
|
|
||||||
`.sofa` 是用户可见的 source of truth;`.jochrtf` 是可删除、可从 SOFA 重建的
|
`.sofa` 是用户可见的 source of truth;`.jochrtf` 是可删除、可从 SOFA 重建的
|
||||||
|
|||||||
Binary file not shown.
@@ -153,6 +153,13 @@ The runtime is fixed at:
|
|||||||
- fifth order, 36 terms, ACN/N3D real spherical harmonics;
|
- fifth order, 36 terms, ACN/N3D real spherical harmonics;
|
||||||
- float64 PCM, delay, SH, and room state; complex128 band transfers and spectra.
|
- float64 PCM, delay, SH, and room state; complex128 band transfers and spectra.
|
||||||
|
|
||||||
|
Those are the SOFA / `.jochrtf` path and the public filterbank. The Rosella
|
||||||
|
`.personalized_headphone` path selects its bank: the public bank above is the
|
||||||
|
default (961 samples), and `--rosella-filterbank 2` switches to the bank captured
|
||||||
|
from the reference renderer (11 taps / 3 slots, 448 samples). Parameters,
|
||||||
|
buffers, latency and validation of both are in
|
||||||
|
[Rosella filterbank](rosella_filterbank.en.md).
|
||||||
|
|
||||||
Real and imaginary unit gains for every hybrid band pass through the same
|
Real and imaginary unit gains for every hybrid band pass through the same
|
||||||
analysis/synthesis chain to form a 154-real-parameter impulse dictionary. The
|
analysis/synthesis chain to form a 154-real-parameter impulse dictionary. The
|
||||||
compiler does not sample 77 FFT bins. Defaults are `1e-3` projection ridge and
|
compiler does not sample 77 FFT bins. Defaults are `1e-3` projection ridge and
|
||||||
|
|||||||
@@ -139,6 +139,12 @@ convention adapter,不能只通过 reshape 进入核心 importer。
|
|||||||
- 五阶、36 项、ACN/N3D real spherical harmonics;
|
- 五阶、36 项、ACN/N3D real spherical harmonics;
|
||||||
- PCM、delay、SH、room state 为 float64;频带传递和频域状态为 complex128。
|
- PCM、delay、SH、room state 为 float64;频带传递和频域状态为 complex128。
|
||||||
|
|
||||||
|
上面是 SOFA / `.jochrtf` 路径与公开滤波器组的固定参数。Rosella
|
||||||
|
`.personalized_headphone` 路径的滤波器组可选:默认仍是这一套公开滤波器组(961
|
||||||
|
samples),`--rosella-filterbank 2` 换成从参考渲染器捕获的版本(11 taps / 3 slots、
|
||||||
|
448 samples)。两套实现的参数、缓冲、延迟与验证见
|
||||||
|
[Rosella 滤波器组](rosella_filterbank.md)。
|
||||||
|
|
||||||
每个 hybrid band 的 real/imaginary 单位增益都通过同一套 analysis/synthesis 链生成
|
每个 hybrid band 的 real/imaginary 单位增益都通过同一套 analysis/synthesis 链生成
|
||||||
脉冲字典,共 154 个实参数;编译不是直接读取 77 个 FFT bin。默认 projection
|
脉冲字典,共 154 个实参数;编译不是直接读取 77 个 FFT bin。默认 projection
|
||||||
ridge 为 `1e-3`,SH ridge 为 `1e-5`。同方向 measurement 先合并,再用球面 Voronoi
|
ridge 为 `1e-3`,SH ridge 为 `1e-5`。同方向 measurement 先合并,再用球面 Voronoi
|
||||||
|
|||||||
@@ -0,0 +1,89 @@
|
|||||||
|
# Rosella filterbank
|
||||||
|
|
||||||
|
[中文](rosella_filterbank.md) · [Back to README](../README.en.md)
|
||||||
|
|
||||||
|
The Rosella `.personalized_headphone` path can run either of two 64-QMF /
|
||||||
|
77-hybrid filterbanks: the shared public bank (the default) or the Rosella bank
|
||||||
|
captured from the reference renderer. The SOFA and `.jochrtf` paths always use
|
||||||
|
the public bank and never consult the selector.
|
||||||
|
|
||||||
|
## Which bank runs
|
||||||
|
|
||||||
|
`--rosella-filterbank 1|2` selects the bank of the Rosella path; the library
|
||||||
|
form is `RosellaBinauralRenderer(..., filterbank=...)`, where omitting the
|
||||||
|
argument means `1`.
|
||||||
|
|
||||||
|
| Value | Implementation | Low-band hybrid | High-band delay | Latency |
|
||||||
|
| --- | --- | --- | --- | ---: |
|
||||||
|
| `1` (default) | `rosella_filterbank.py` | 13 taps | 6 slots | 961 samples |
|
||||||
|
| `2` | `rosella_captured_filterbank.py` | 11 taps | 3 slots | 448 samples |
|
||||||
|
|
||||||
|
Both banks expose the same four stateful half-steps (QMF analysis, hybrid
|
||||||
|
analysis, hybrid synthesis, QMF synthesis), so the choice switches the
|
||||||
|
coefficients, the high-band delay, the history layout, the startup skip and the
|
||||||
|
flush capacity together. The analysis and synthesis stages keep their own state
|
||||||
|
and never share it. Any other value is rejected, and the option is an error when
|
||||||
|
the binaural input is not a Rosella model: a SOFA or `.jochrtf` render must not
|
||||||
|
silently ignore it.
|
||||||
|
|
||||||
|
## Why the banks differ
|
||||||
|
|
||||||
|
The reference headphone renderer selects its analysis and synthesis callbacks
|
||||||
|
during preparation, after initialization. Capturing only the initial callbacks
|
||||||
|
identified the wrong filterbank variant. Runtime buffer captures gave these
|
||||||
|
parameters:
|
||||||
|
|
||||||
|
| Parameter | Captured Rosella bank | Public bank |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| QMF prototype | 640 asymmetric coefficients | polyphase reorder of the public 640-tap prototype |
|
||||||
|
| Low-band hybrid filter | 11 taps | 13 taps |
|
||||||
|
| High-band hybrid delay | 3 slots | 6 slots |
|
||||||
|
| Analysis/synthesis latency | 448 samples | 961 samples |
|
||||||
|
|
||||||
|
The wrong variant produced a frequency-dependent room-tail error. With the
|
||||||
|
captured room send as input, the existing room recursion already matched the
|
||||||
|
reference, so the fix changes only the filterbank: not the room coefficients,
|
||||||
|
the direct HRTF calculation, the phase mapping, or the output EQ. The public
|
||||||
|
bank stayed as bank `1` so historical renders remain reproducible; the captured
|
||||||
|
bank is bank `2`.
|
||||||
|
|
||||||
|
## Implementation
|
||||||
|
|
||||||
|
`rosella_captured_filterbank.py` reproduces the four half-steps of the captured
|
||||||
|
bank in float64/complex128:
|
||||||
|
|
||||||
|
- QMF analysis keeps a 9-slot ring history, reads `(position + tap + 1) mod 10`,
|
||||||
|
and writes `spectrum[2b] = F[b]/64`, `spectrum[2b+1] = conj(F[63-b])/64`;
|
||||||
|
- hybrid analysis keeps a 10-slot history, transforms the three low bands with
|
||||||
|
8/4/4-point transforms, reorders them with a fixed map, conjugates bands 1 and
|
||||||
|
3, and passes the high bands through delayed by 3 slots;
|
||||||
|
- QMF synthesis keeps a 9-slot history and accumulates 10 prototype-matched
|
||||||
|
lags, taking the imaginary part on odd lags.
|
||||||
|
|
||||||
|
`data/rosella_captured_kernels.npz` stores the captured prototype
|
||||||
|
(`float32[64,10]`), rotation (`complex64[64]`) and hybrid coefficients
|
||||||
|
(`complex64[22]`); they are promoted to float64/complex128 on load. The values
|
||||||
|
are the ones the C++ core compiles in as `src/hrtf/rosella_filterbank_tables.h`
|
||||||
|
and contain no personalized HRTF or room tuning.
|
||||||
|
|
||||||
|
The startup skip and the flush capacity both come from the selected bank (961 or
|
||||||
|
448 samples). `.report.json` records the actual choice in
|
||||||
|
`binaural_renderer_backend`, as `filterbank`, `filterbank_name` and
|
||||||
|
`latency_compensated_samples`.
|
||||||
|
|
||||||
|
## Validation
|
||||||
|
|
||||||
|
The local regression script lives in `tests/`, which this repository does not
|
||||||
|
track (like the rest of its local checks). It covers:
|
||||||
|
|
||||||
|
- reference vectors for the four captured half-steps, the same samples the C++
|
||||||
|
core checks, generated by an independent NumPy implementation, to within
|
||||||
|
`1e-10` per sample;
|
||||||
|
- an impulse roundtrip peak at 961 and 448 samples for the two banks;
|
||||||
|
- chunk invariance (including zero-length and unequal chunks) and `reset()`;
|
||||||
|
- independence: processing one bank never changes the other bank's state;
|
||||||
|
- rejection of out-of-range values.
|
||||||
|
|
||||||
|
The public filterbank formulas and `.jochrtf` direction field described in
|
||||||
|
`docs/binaural.md` are unchanged; the captured bank only affects the Rosella
|
||||||
|
path.
|
||||||
@@ -0,0 +1,71 @@
|
|||||||
|
# Rosella 滤波器组
|
||||||
|
|
||||||
|
[English](rosella_filterbank.en.md) · [返回 README](../README.md)
|
||||||
|
|
||||||
|
Rosella `.personalized_headphone` 路径可以在两套 64-QMF / 77-hybrid 滤波器组之间选择:
|
||||||
|
公开滤波器组(默认)与从参考渲染器捕获的 Rosella 滤波器组。SOFA 与 `.jochrtf` 路径
|
||||||
|
始终使用公开滤波器组,不经过该选择。
|
||||||
|
|
||||||
|
## 选择哪一套
|
||||||
|
|
||||||
|
`--rosella-filterbank 1|2` 选择 Rosella 路径的滤波器组;库接口是
|
||||||
|
`RosellaBinauralRenderer(..., filterbank=...)`,省略参数等于 `1`。
|
||||||
|
|
||||||
|
| 取值 | 实现 | 低频 hybrid | 高频 slot delay | 延迟 |
|
||||||
|
| --- | --- | --- | --- | ---: |
|
||||||
|
| `1`(默认) | `rosella_filterbank.py` | 13 taps | 6 slots | 961 samples |
|
||||||
|
| `2` | `rosella_captured_filterbank.py` | 11 taps | 3 slots | 448 samples |
|
||||||
|
|
||||||
|
两套实现暴露同一组四个有状态的 half-step(QMF 分析、hybrid 分析、hybrid 合成、
|
||||||
|
QMF 合成),因此切换会同时改变系数、高频延迟、历史缓冲布局、起始延迟跳过量和
|
||||||
|
flush 容量;分析与合成两级各自持有状态,互不共享。其它取值直接报错;非 Rosella
|
||||||
|
输入(SOFA、`.jochrtf`)给出该选项也报错,不会静默忽略。
|
||||||
|
|
||||||
|
## 为什么两套不同
|
||||||
|
|
||||||
|
参考头戴渲染器在初始化之后的准备阶段才选定 analysis/synthesis 回调,只捕获初始化
|
||||||
|
时的回调会拿到错误的滤波器组变体。运行时缓冲捕获得到的参数是:
|
||||||
|
|
||||||
|
| 参数 | Rosella 捕获版 | 公开版 |
|
||||||
|
| --- | --- | --- |
|
||||||
|
| QMF prototype | 640 个非对称系数 | 公开 640-tap prototype 的多相重排 |
|
||||||
|
| 低频 hybrid 滤波 | 11 taps | 13 taps |
|
||||||
|
| 高频 hybrid delay | 3 slots | 6 slots |
|
||||||
|
| analysis/synthesis 延迟 | 448 samples | 961 samples |
|
||||||
|
|
||||||
|
用错变体会在 room 尾音上引入随频率变化的误差;把捕获到的 room send 作为输入时,
|
||||||
|
现有 room 递推已经能对上参考,因此修复只更换滤波器组,不改 room 系数、direct HRTF、
|
||||||
|
相位映射或输出 EQ。公开版是 `1`,捕获版是 `2`;默认保留公开版,便于与历史输出对齐。
|
||||||
|
|
||||||
|
## 实现
|
||||||
|
|
||||||
|
`rosella_captured_filterbank.py` 用 float64/complex128 复现捕获版的四个 half-step:
|
||||||
|
|
||||||
|
- QMF 分析保留 9 slot 环形历史,按 `(position + tap + 1) mod 10` 读取,输出
|
||||||
|
`spectrum[2b] = F[b]/64`、`spectrum[2b+1] = conj(F[63-b])/64`;
|
||||||
|
- hybrid 分析保留 10 slot 历史,低频三个 band 分别做 8/4/4 点变换后按固定顺序重排,
|
||||||
|
band 1 与 band 3 取共轭;高频带原样延迟 3 slot;
|
||||||
|
- QMF 合成保留 9 slot 历史,用 10 个 lag 的 prototype 匹配累加,奇数 lag 取虚部。
|
||||||
|
|
||||||
|
`data/rosella_captured_kernels.npz` 保存捕获版的 prototype(`float32[64,10]`)、
|
||||||
|
rotation(`complex64[64]`)与 hybrid 系数(`complex64[22]`);载入后提升为
|
||||||
|
float64/complex128。这些数值与 C++ 内核编译进去的
|
||||||
|
`src/hrtf/rosella_filterbank_tables.h` 相同,不包含个性化 HRTF 或 room 调参。
|
||||||
|
|
||||||
|
运行时的起始延迟跳过与 flush 容量都取自所选滤波器组:`961` 或 `448` samples,
|
||||||
|
`.report.json` 的 `binaural_renderer_backend` 用 `filterbank`、`filterbank_name` 与
|
||||||
|
`latency_compensated_samples` 记录实际选择。
|
||||||
|
|
||||||
|
## 验证
|
||||||
|
|
||||||
|
本地回归脚本放在 `tests/`(该目录不随仓库跟踪,与其余本地验证脚本一致),覆盖:
|
||||||
|
|
||||||
|
- 捕获版四个 half-step 的参考向量(与 C++ 内核 `tests/rosella_filterbank_vectors.h`
|
||||||
|
同一组、由独立 NumPy 实现生成),逐样本误差小于 `1e-10`;
|
||||||
|
- 两套滤波器组的脉冲往返延迟分别落在 `961` 与 `448` samples;
|
||||||
|
- 分块处理(含 0 长度与不等长切分)与 `reset()` 不变性;
|
||||||
|
- 两套滤波器组互不干扰:交叉处理一个 bank 不改变另一个 bank 的状态;
|
||||||
|
- 越界取值被拒绝。
|
||||||
|
|
||||||
|
`docs/binaural.md` 描述的公开滤波器组公式与 `.jochrtf` 方向和公开版一致;捕获版只
|
||||||
|
作用于 Rosella 路径。
|
||||||
@@ -38,10 +38,10 @@ from binaural_renderer import (
|
|||||||
from rosella_binaural_renderer import (
|
from rosella_binaural_renderer import (
|
||||||
DEFAULT_PERSONALIZED_HEADPHONE,
|
DEFAULT_PERSONALIZED_HEADPHONE,
|
||||||
ROSSELLA_BLOCK_SAMPLES,
|
ROSSELLA_BLOCK_SAMPLES,
|
||||||
ROSSELLA_LATENCY_SAMPLES,
|
|
||||||
RosellaBinauralRenderer,
|
RosellaBinauralRenderer,
|
||||||
resolve_personalized_headphone,
|
resolve_personalized_headphone,
|
||||||
)
|
)
|
||||||
|
from rosella_bank import DEFAULT_FILTERBANK, FILTERBANK_CHOICES, FILTERBANK_NAMES
|
||||||
from sofa_hrtf_field import DEFAULT_HRTF_CACHE_DIR
|
from sofa_hrtf_field import DEFAULT_HRTF_CACHE_DIR
|
||||||
from speaker_backend import create_speaker_renderer
|
from speaker_backend import create_speaker_renderer
|
||||||
from speaker_layouts import (SPEAKER_LAYOUT_CHOICES, get_speaker_layout,
|
from speaker_layouts import (SPEAKER_LAYOUT_CHOICES, get_speaker_layout,
|
||||||
@@ -566,6 +566,10 @@ def build_parser():
|
|||||||
const=DEFAULT_PERSONALIZED_HEADPHONE,
|
const=DEFAULT_PERSONALIZED_HEADPHONE,
|
||||||
help="Rosella .personalized_headphone 模型;不带路径时默认 "
|
help="Rosella .personalized_headphone 模型;不带路径时默认 "
|
||||||
"HRTF/binaural.personalized_headphone")
|
"HRTF/binaural.personalized_headphone")
|
||||||
|
parser.add_argument(
|
||||||
|
"--rosella-filterbank", type=int, choices=FILTERBANK_CHOICES, default=None,
|
||||||
|
help="Rosella 模型的滤波器组:1=13 taps/6 slots(默认),"
|
||||||
|
"2=11 taps/3 slots;仅作用于个性化模型")
|
||||||
parser.add_argument(
|
parser.add_argument(
|
||||||
"--hrtf-cache-policy", choices=("none", "memory", "disk"), default=None,
|
"--hrtf-cache-policy", choices=("none", "memory", "disk"), default=None,
|
||||||
help="SOFA 编译缓存;默认 memory,disk 写入可删除的 .jochrtf")
|
help="SOFA 编译缓存;默认 memory,disk 写入可删除的 .jochrtf")
|
||||||
@@ -653,6 +657,10 @@ def main(argv=None):
|
|||||||
))
|
))
|
||||||
if hrtf_options_used and not binaural_mode:
|
if hrtf_options_used and not binaural_mode:
|
||||||
raise ValueError("SOFA/HRTF 选项仅与 --binaural 一起使用")
|
raise ValueError("SOFA/HRTF 选项仅与 --binaural 一起使用")
|
||||||
|
if args.rosella_filterbank is not None and not binaural_mode:
|
||||||
|
raise ValueError("--rosella-filterbank 仅与 --binaural 一起使用")
|
||||||
|
rosella_filterbank = (DEFAULT_FILTERBANK if args.rosella_filterbank is None
|
||||||
|
else args.rosella_filterbank)
|
||||||
if (not math.isfinite(args.binaural_tail_seconds)
|
if (not math.isfinite(args.binaural_tail_seconds)
|
||||||
or args.binaural_tail_seconds < 0):
|
or args.binaural_tail_seconds < 0):
|
||||||
raise ValueError("binaural-tail-seconds 必须是非负有限值")
|
raise ValueError("binaural-tail-seconds 必须是非负有限值")
|
||||||
@@ -686,6 +694,11 @@ def main(argv=None):
|
|||||||
raise ValueError("eac3-target-level 必须在 -31..0 之间")
|
raise ValueError("eac3-target-level 必须在 -31..0 之间")
|
||||||
binaural_hrtf_input = resolve_binaural_hrtf_input(
|
binaural_hrtf_input = resolve_binaural_hrtf_input(
|
||||||
args, required=binaural_mode and not args.metadata_only)
|
args, required=binaural_mode and not args.metadata_only)
|
||||||
|
if (args.rosella_filterbank is not None and binaural_hrtf_input is not None
|
||||||
|
and binaural_hrtf_input["kind"] != "rosella"):
|
||||||
|
raise ValueError(
|
||||||
|
"--rosella-filterbank 只作用于 Rosella .personalized_headphone 输入;"
|
||||||
|
"当前 SOFA/.jochrtf 路径不使用该滤波器组")
|
||||||
ffmpeg = executable(args.ffmpeg, "FFmpeg")
|
ffmpeg = executable(args.ffmpeg, "FFmpeg")
|
||||||
decode_options = ()
|
decode_options = ()
|
||||||
decode_option_info = None
|
decode_option_info = None
|
||||||
@@ -828,6 +841,7 @@ def main(argv=None):
|
|||||||
hrtf_source["path"],
|
hrtf_source["path"],
|
||||||
bed_channels=getattr(index, "bed_channels", ()),
|
bed_channels=getattr(index, "bed_channels", ()),
|
||||||
mode=binaural_render_mode,
|
mode=binaural_render_mode,
|
||||||
|
filterbank=rosella_filterbank,
|
||||||
object_delay_samples=args.object_delay_samples,
|
object_delay_samples=args.object_delay_samples,
|
||||||
tail_seconds=args.binaural_tail_seconds,
|
tail_seconds=args.binaural_tail_seconds,
|
||||||
output_gain=gain_float64,
|
output_gain=gain_float64,
|
||||||
@@ -844,9 +858,12 @@ def main(argv=None):
|
|||||||
f"precision=float64/complex128 "
|
f"precision=float64/complex128 "
|
||||||
f"hrtf={hrtf_source['kind']}:{hrtf_source['path']}", flush=True)
|
f"hrtf={hrtf_source['kind']}:{hrtf_source['path']}", flush=True)
|
||||||
if hrtf_source["kind"] == "rosella":
|
if hrtf_source["kind"] == "rosella":
|
||||||
|
print(
|
||||||
|
f"[binaural] filterbank={FILTERBANK_NAMES[binaural_decoder.filterbank]}"
|
||||||
|
f" latency={binaural_decoder.latency_samples}", flush=True)
|
||||||
flush_samples = math.ceil(
|
flush_samples = math.ceil(
|
||||||
(args.binaural_tail_seconds * RATE
|
(args.binaural_tail_seconds * RATE
|
||||||
+ ROSSELLA_LATENCY_SAMPLES + ROSSELLA_BLOCK_SAMPLES)
|
+ binaural_decoder.latency_samples + ROSSELLA_BLOCK_SAMPLES)
|
||||||
/ ROSSELLA_BLOCK_SAMPLES) * ROSSELLA_BLOCK_SAMPLES
|
/ ROSSELLA_BLOCK_SAMPLES) * ROSSELLA_BLOCK_SAMPLES
|
||||||
spool_capacity = frame_count * FRAME_SAMPLES + flush_samples
|
spool_capacity = frame_count * FRAME_SAMPLES + flush_samples
|
||||||
else:
|
else:
|
||||||
|
|||||||
@@ -0,0 +1,101 @@
|
|||||||
|
"""Selectable Rosella 64-QMF / 77-hybrid filterbank.
|
||||||
|
|
||||||
|
The Rosella renderer can run either bank:
|
||||||
|
|
||||||
|
``PUBLIC_FILTERBANK`` (1)
|
||||||
|
The shared public bank of ``rosella_filterbank.py``: a 13-tap hybrid
|
||||||
|
analysis, a 6-slot high-band delay and 961 samples of analysis/synthesis
|
||||||
|
latency. This is the bank the reference implementation has always used.
|
||||||
|
|
||||||
|
``CAPTURED_FILTERBANK`` (2)
|
||||||
|
The bank the reference headphone renderer actually runs, captured from it:
|
||||||
|
an 11-tap hybrid analysis, a 3-slot high-band delay and 448 samples of
|
||||||
|
latency.
|
||||||
|
|
||||||
|
Both banks expose the same four stateful stages, so the choice changes the
|
||||||
|
coefficients, the high-band delay, the history layout and the latency together.
|
||||||
|
Nothing is shared between them.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from rosella_captured_filterbank import (
|
||||||
|
DEFAULT_CAPTURED_KERNEL_DATA,
|
||||||
|
HybridAnalysis as CapturedHybridAnalysis,
|
||||||
|
HybridSynthesis as CapturedHybridSynthesis,
|
||||||
|
QmfAnalysis as CapturedQmfAnalysis,
|
||||||
|
QmfSynthesis as CapturedQmfSynthesis,
|
||||||
|
)
|
||||||
|
from rosella_filterbank import (
|
||||||
|
DEFAULT_KERNEL_DATA,
|
||||||
|
HybridAnalysis as PublicHybridAnalysis,
|
||||||
|
HybridSynthesis as PublicHybridSynthesis,
|
||||||
|
QmfAnalysis as PublicQmfAnalysis,
|
||||||
|
QmfSynthesis as PublicQmfSynthesis,
|
||||||
|
)
|
||||||
|
|
||||||
|
PUBLIC_FILTERBANK = 1
|
||||||
|
CAPTURED_FILTERBANK = 2
|
||||||
|
FILTERBANK_CHOICES = (PUBLIC_FILTERBANK, CAPTURED_FILTERBANK)
|
||||||
|
DEFAULT_FILTERBANK = PUBLIC_FILTERBANK
|
||||||
|
|
||||||
|
FILTERBANK_NAMES = {
|
||||||
|
PUBLIC_FILTERBANK: "public",
|
||||||
|
CAPTURED_FILTERBANK: "captured",
|
||||||
|
}
|
||||||
|
|
||||||
|
# analysis/synthesis latency of each bank, in samples at 48 kHz
|
||||||
|
FILTERBANK_LATENCY_SAMPLES = {
|
||||||
|
PUBLIC_FILTERBANK: 961,
|
||||||
|
CAPTURED_FILTERBANK: 448,
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def latency_samples(filterbank: int) -> int:
|
||||||
|
"""Length of the startup transient the runtime removes for ``filterbank``."""
|
||||||
|
try:
|
||||||
|
return FILTERBANK_LATENCY_SAMPLES[int(filterbank)]
|
||||||
|
except (KeyError, TypeError, ValueError):
|
||||||
|
raise ValueError(
|
||||||
|
f"Rosella filterbank must be 1 (public) or 2 (captured), got {filterbank!r}"
|
||||||
|
) from None
|
||||||
|
|
||||||
|
|
||||||
|
class RosellaBank:
|
||||||
|
"""The four stages of one bank, each with its own state."""
|
||||||
|
|
||||||
|
def __init__(self, filterbank: int = DEFAULT_FILTERBANK, *,
|
||||||
|
source_channels: int, output_channels: int,
|
||||||
|
kernel_data: str | Path | None = None):
|
||||||
|
if filterbank not in FILTERBANK_CHOICES:
|
||||||
|
raise ValueError(
|
||||||
|
f"Rosella filterbank must be 1 (public) or 2 (captured), got {filterbank!r}")
|
||||||
|
kind = int(filterbank)
|
||||||
|
if source_channels <= 0 or output_channels <= 0:
|
||||||
|
raise ValueError("channel counts must be positive")
|
||||||
|
self.filterbank = kind
|
||||||
|
self.name = FILTERBANK_NAMES[kind]
|
||||||
|
self.latency_samples = latency_samples(kind)
|
||||||
|
if kind == PUBLIC_FILTERBANK:
|
||||||
|
self.table_path = Path(
|
||||||
|
DEFAULT_KERNEL_DATA if kernel_data is None else kernel_data
|
||||||
|
).expanduser().resolve()
|
||||||
|
self.qmf_analysis = PublicQmfAnalysis(source_channels, self.table_path)
|
||||||
|
self.hybrid_analysis = PublicHybridAnalysis(source_channels, self.table_path)
|
||||||
|
self.hybrid_synthesis = PublicHybridSynthesis(output_channels, self.table_path)
|
||||||
|
self.qmf_synthesis = PublicQmfSynthesis(output_channels, self.table_path)
|
||||||
|
else:
|
||||||
|
self.table_path = Path(
|
||||||
|
DEFAULT_CAPTURED_KERNEL_DATA if kernel_data is None else kernel_data
|
||||||
|
).expanduser().resolve()
|
||||||
|
self.qmf_analysis = CapturedQmfAnalysis(source_channels, self.table_path)
|
||||||
|
self.hybrid_analysis = CapturedHybridAnalysis(source_channels, self.table_path)
|
||||||
|
self.hybrid_synthesis = CapturedHybridSynthesis(output_channels, self.table_path)
|
||||||
|
self.qmf_synthesis = CapturedQmfSynthesis(output_channels, self.table_path)
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.qmf_analysis.reset()
|
||||||
|
self.hybrid_analysis.reset()
|
||||||
|
self.hybrid_synthesis.reset()
|
||||||
|
self.qmf_synthesis.reset()
|
||||||
@@ -16,22 +16,16 @@ import numpy as np
|
|||||||
|
|
||||||
from bed_layout import bed_object_positions
|
from bed_layout import bed_object_positions
|
||||||
from binaural_metadata import OamdPositionTimeline
|
from binaural_metadata import OamdPositionTimeline
|
||||||
|
from rosella_bank import (DEFAULT_FILTERBANK, FILTERBANK_CHOICES, FILTERBANK_NAMES,
|
||||||
|
RosellaBank)
|
||||||
from rosella_core import RosellaRenderer
|
from rosella_core import RosellaRenderer
|
||||||
from rosella_direct import BINAURAL_PROFILE_NAMES
|
from rosella_direct import BINAURAL_PROFILE_NAMES
|
||||||
from rosella_filterbank import (
|
|
||||||
DEFAULT_KERNEL_DATA,
|
|
||||||
HybridAnalysis,
|
|
||||||
HybridSynthesis,
|
|
||||||
QmfAnalysis,
|
|
||||||
QmfSynthesis,
|
|
||||||
)
|
|
||||||
from rosella_model import RosellaModel, load_personalized_headphone
|
from rosella_model import RosellaModel, load_personalized_headphone
|
||||||
|
|
||||||
SAMPLE_RATE = 48000
|
SAMPLE_RATE = 48000
|
||||||
FRAME_SAMPLES = 1536
|
FRAME_SAMPLES = 1536
|
||||||
ROSSELLA_BLOCK_SAMPLES = 512
|
ROSSELLA_BLOCK_SAMPLES = 512
|
||||||
QMF_HOP_SAMPLES = 64
|
QMF_HOP_SAMPLES = 64
|
||||||
ROSSELLA_LATENCY_SAMPLES = 961
|
|
||||||
SOURCE_CHANNELS = 16
|
SOURCE_CHANNELS = 16
|
||||||
OUTPUT_CHANNELS = 2
|
OUTPUT_CHANNELS = 2
|
||||||
PROJECT_DIR = Path(__file__).resolve().parent.parent
|
PROJECT_DIR = Path(__file__).resolve().parent.parent
|
||||||
@@ -67,7 +61,8 @@ class RosellaBinauralRenderer:
|
|||||||
personalized_headphone: str | Path | RosellaModel,
|
personalized_headphone: str | Path | RosellaModel,
|
||||||
*,
|
*,
|
||||||
mode: str = "mid",
|
mode: str = "mid",
|
||||||
kernel_data: str | Path = DEFAULT_KERNEL_DATA,
|
filterbank: int = DEFAULT_FILTERBANK,
|
||||||
|
kernel_data: str | Path | None = None,
|
||||||
object_delay_samples: int = 1473,
|
object_delay_samples: int = 1473,
|
||||||
tail_seconds: float = 5.0,
|
tail_seconds: float = 5.0,
|
||||||
output_gain: float = 1.0,
|
output_gain: float = 1.0,
|
||||||
@@ -76,6 +71,9 @@ class RosellaBinauralRenderer:
|
|||||||
bed_channels=None):
|
bed_channels=None):
|
||||||
if mode not in BINAURAL_PROFILE_NAMES:
|
if mode not in BINAURAL_PROFILE_NAMES:
|
||||||
raise ValueError("binaural mode must be near, mid, or far")
|
raise ValueError("binaural mode must be near, mid, or far")
|
||||||
|
if filterbank not in FILTERBANK_CHOICES:
|
||||||
|
raise ValueError(
|
||||||
|
f"Rosella filterbank must be 1 (public) or 2 (captured), got {filterbank!r}")
|
||||||
if int(object_delay_samples) < 0:
|
if int(object_delay_samples) < 0:
|
||||||
raise ValueError("object_delay_samples must be non-negative")
|
raise ValueError("object_delay_samples must be non-negative")
|
||||||
if float(tail_seconds) < 0.0:
|
if float(tail_seconds) < 0.0:
|
||||||
@@ -97,8 +95,6 @@ class RosellaBinauralRenderer:
|
|||||||
|
|
||||||
self.mode = mode
|
self.mode = mode
|
||||||
self.profile_index = BINAURAL_PROFILE_NAMES[mode]
|
self.profile_index = BINAURAL_PROFILE_NAMES[mode]
|
||||||
self.kernel_data = Path(kernel_data).expanduser().resolve()
|
|
||||||
self.kernel_data_sha256 = _sha256_file(self.kernel_data)
|
|
||||||
self.object_delay_samples = int(object_delay_samples)
|
self.object_delay_samples = int(object_delay_samples)
|
||||||
self.tail_seconds = float(tail_seconds)
|
self.tail_seconds = float(tail_seconds)
|
||||||
self.output_gain = np.float64(output_gain)
|
self.output_gain = np.float64(output_gain)
|
||||||
@@ -106,13 +102,21 @@ class RosellaBinauralRenderer:
|
|||||||
self.chunk_samples = self.chunk_frames * FRAME_SAMPLES
|
self.chunk_samples = self.chunk_frames * FRAME_SAMPLES
|
||||||
|
|
||||||
self.dsp_backend = "python"
|
self.dsp_backend = "python"
|
||||||
self.qmf_analysis = QmfAnalysis(SOURCE_CHANNELS, self.kernel_data)
|
self.bank = RosellaBank(
|
||||||
self.hybrid_analysis = HybridAnalysis(SOURCE_CHANNELS, self.kernel_data)
|
filterbank, source_channels=SOURCE_CHANNELS,
|
||||||
|
output_channels=OUTPUT_CHANNELS, kernel_data=kernel_data)
|
||||||
|
self.filterbank = self.bank.filterbank
|
||||||
|
self.filterbank_name = FILTERBANK_NAMES[self.filterbank]
|
||||||
|
self.latency_samples = self.bank.latency_samples
|
||||||
|
self.kernel_data = self.bank.table_path
|
||||||
|
self.kernel_data_sha256 = _sha256_file(self.kernel_data)
|
||||||
|
self.qmf_analysis = self.bank.qmf_analysis
|
||||||
|
self.hybrid_analysis = self.bank.hybrid_analysis
|
||||||
self.core = RosellaRenderer(
|
self.core = RosellaRenderer(
|
||||||
self.model, SOURCE_CHANNELS,
|
self.model, SOURCE_CHANNELS,
|
||||||
room_impulse_slots=room_impulse_slots)
|
room_impulse_slots=room_impulse_slots)
|
||||||
self.hybrid_synthesis = HybridSynthesis(OUTPUT_CHANNELS, self.kernel_data)
|
self.hybrid_synthesis = self.bank.hybrid_synthesis
|
||||||
self.qmf_synthesis = QmfSynthesis(OUTPUT_CHANNELS, self.kernel_data)
|
self.qmf_synthesis = self.bank.qmf_synthesis
|
||||||
self.timeline = OamdPositionTimeline(
|
self.timeline = OamdPositionTimeline(
|
||||||
15, bed_positions=bed_object_positions(bed_channels or ()))
|
15, bed_positions=bed_object_positions(bed_channels or ()))
|
||||||
|
|
||||||
@@ -213,7 +217,7 @@ class RosellaBinauralRenderer:
|
|||||||
stereo *= self.output_gain
|
stereo *= self.output_gain
|
||||||
|
|
||||||
skip = max(0, min(
|
skip = max(0, min(
|
||||||
len(stereo), ROSSELLA_LATENCY_SAMPLES - self.raw_output_samples))
|
len(stereo), self.latency_samples - self.raw_output_samples))
|
||||||
self.raw_output_samples += len(stereo)
|
self.raw_output_samples += len(stereo)
|
||||||
self.processed_input_samples += len(values)
|
self.processed_input_samples += len(values)
|
||||||
output = stereo[skip:]
|
output = stereo[skip:]
|
||||||
@@ -231,7 +235,7 @@ class RosellaBinauralRenderer:
|
|||||||
self._buffer_used = 0
|
self._buffer_used = 0
|
||||||
flush_samples = math.ceil(
|
flush_samples = math.ceil(
|
||||||
(self.tail_seconds * SAMPLE_RATE
|
(self.tail_seconds * SAMPLE_RATE
|
||||||
+ ROSSELLA_LATENCY_SAMPLES + ROSSELLA_BLOCK_SAMPLES)
|
+ self.latency_samples + ROSSELLA_BLOCK_SAMPLES)
|
||||||
/ ROSSELLA_BLOCK_SAMPLES) * ROSSELLA_BLOCK_SAMPLES
|
/ ROSSELLA_BLOCK_SAMPLES) * ROSSELLA_BLOCK_SAMPLES
|
||||||
while flush_samples:
|
while flush_samples:
|
||||||
count = min(flush_samples, self.chunk_samples)
|
count = min(flush_samples, self.chunk_samples)
|
||||||
@@ -259,7 +263,9 @@ class RosellaBinauralRenderer:
|
|||||||
"kernel_data": str(self.kernel_data),
|
"kernel_data": str(self.kernel_data),
|
||||||
"kernel_data_sha256": self.kernel_data_sha256,
|
"kernel_data_sha256": self.kernel_data_sha256,
|
||||||
"mode": self.mode,
|
"mode": self.mode,
|
||||||
"latency_compensated_samples": ROSSELLA_LATENCY_SAMPLES,
|
"filterbank": self.filterbank,
|
||||||
|
"filterbank_name": self.filterbank_name,
|
||||||
|
"latency_compensated_samples": self.latency_samples,
|
||||||
"object_delay_samples": self.object_delay_samples,
|
"object_delay_samples": self.object_delay_samples,
|
||||||
"tail_seconds": self.tail_seconds,
|
"tail_seconds": self.tail_seconds,
|
||||||
"metadata_payloads": self.timeline.payload_count,
|
"metadata_payloads": self.timeline.payload_count,
|
||||||
|
|||||||
@@ -0,0 +1,218 @@
|
|||||||
|
"""Captured Rosella 64-QMF / 77-hybrid filterbank.
|
||||||
|
|
||||||
|
The Rosella headphone renderer selects its analysis and synthesis callbacks
|
||||||
|
during preparation, after initialization. The callbacks it actually runs are
|
||||||
|
this bank: a 640-coefficient QMF prototype, an 11-tap hybrid analysis with a
|
||||||
|
3-slot high-band delay, and a 448-sample analysis/synthesis latency. The shared
|
||||||
|
public bank lives in ``rosella_filterbank.py`` and differs in every one of those
|
||||||
|
parameters (13-tap hybrid, 6-slot delay, 961-sample latency).
|
||||||
|
|
||||||
|
The tables are the float32 values the C++ core compiles in as
|
||||||
|
``src/hrtf/rosella_filterbank_tables.h``; ``data/rosella_captured_kernels.npz``
|
||||||
|
holds the same numbers. State is float64/complex128, like the rest of the
|
||||||
|
reference implementation.
|
||||||
|
"""
|
||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from functools import lru_cache
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import numpy as np
|
||||||
|
|
||||||
|
PROJECT_DIR = Path(__file__).resolve().parent.parent
|
||||||
|
DEFAULT_CAPTURED_KERNEL_DATA = PROJECT_DIR / "data" / "rosella_captured_kernels.npz"
|
||||||
|
|
||||||
|
QMF_BANDS = 64
|
||||||
|
HYBRID_BANDS = 77
|
||||||
|
QMF_TAPS = 10
|
||||||
|
HYBRID_TAPS = 11
|
||||||
|
HYBRID_DELAY = 3
|
||||||
|
HYBRID_LOW_BANDS = 16
|
||||||
|
ANALYSIS_SYNTHESIS_LATENCY_SAMPLES = 448
|
||||||
|
|
||||||
|
# The captured renderer's hybrid band order: destination[k] = low[HYBRID_ORDER[k]].
|
||||||
|
HYBRID_ORDER = (0, 7, 1, 6, 2, 9, 3, 4, 10, 5, 11, 15, 8, 12, 13, 14)
|
||||||
|
|
||||||
|
# Bands 1 and 3 are conjugated after the permutation.
|
||||||
|
_CONJUGATED_BANDS = (1, 3)
|
||||||
|
|
||||||
|
|
||||||
|
def _tap_sign(tap: int) -> float:
|
||||||
|
return 1.0 if (tap // 2) % 2 == 0 else -1.0
|
||||||
|
|
||||||
|
|
||||||
|
@lru_cache(maxsize=4)
|
||||||
|
def _load_tables(path_string: str) -> dict[str, np.ndarray]:
|
||||||
|
path = Path(path_string)
|
||||||
|
if not path.is_file():
|
||||||
|
raise FileNotFoundError(f"Rosella captured kernel data not found: {path}")
|
||||||
|
with np.load(path, allow_pickle=False) as archive:
|
||||||
|
version = archive["format_version"]
|
||||||
|
if version.shape != (1,) or int(version[0]) != 1:
|
||||||
|
raise ValueError(
|
||||||
|
f"unsupported Rosella captured kernel data version in {path}")
|
||||||
|
prototype = np.asarray(archive["qmf_prototype"], dtype=np.float64)
|
||||||
|
rotation = np.asarray(archive["rotation"], dtype=np.complex128)
|
||||||
|
hybrid = np.asarray(archive["hybrid"], dtype=np.complex128)
|
||||||
|
if prototype.shape != (QMF_BANDS, QMF_TAPS):
|
||||||
|
raise ValueError("invalid qmf_prototype shape")
|
||||||
|
if rotation.shape != (QMF_BANDS,):
|
||||||
|
raise ValueError("invalid rotation shape")
|
||||||
|
if hybrid.shape != (2 * HYBRID_TAPS,):
|
||||||
|
raise ValueError("invalid hybrid shape")
|
||||||
|
return {"qmf_prototype": prototype, "rotation": rotation, "hybrid": hybrid}
|
||||||
|
|
||||||
|
|
||||||
|
def load_kernel_tables(path: str | Path = DEFAULT_CAPTURED_KERNEL_DATA
|
||||||
|
) -> dict[str, np.ndarray]:
|
||||||
|
"""Load and cache the captured Rosella kernel tables."""
|
||||||
|
return _load_tables(str(Path(path).expanduser().resolve()))
|
||||||
|
|
||||||
|
|
||||||
|
class QmfAnalysis:
|
||||||
|
"""64-band analysis: 640-coefficient prototype, 9-slot chronological history."""
|
||||||
|
|
||||||
|
def __init__(self, channels: int, kernel_data: str | Path = DEFAULT_CAPTURED_KERNEL_DATA):
|
||||||
|
if channels <= 0:
|
||||||
|
raise ValueError("channels must be positive")
|
||||||
|
tables = load_kernel_tables(kernel_data)
|
||||||
|
self.prototype = tables["qmf_prototype"]
|
||||||
|
self.rotation = tables["rotation"]
|
||||||
|
self.channels = int(channels)
|
||||||
|
self.history = np.zeros((QMF_TAPS - 1, self.channels, QMF_BANDS),
|
||||||
|
dtype=np.float64)
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.history.fill(0.0)
|
||||||
|
|
||||||
|
def process_chunk(self, hops) -> np.ndarray:
|
||||||
|
values = np.asarray(hops, dtype=np.float64)
|
||||||
|
if values.ndim != 3 or values.shape[1:] != (self.channels, QMF_BANDS):
|
||||||
|
raise ValueError(f"expected [slots,{self.channels},64], got {values.shape}")
|
||||||
|
count = values.shape[0]
|
||||||
|
joined = np.concatenate((self.history, values), axis=0)
|
||||||
|
even = np.zeros_like(values)
|
||||||
|
odd = np.zeros_like(values)
|
||||||
|
for tap in range(QMF_TAPS):
|
||||||
|
# The window of output i is joined[i:i+10], oldest tap first.
|
||||||
|
source = joined[tap:tap + count]
|
||||||
|
target = even if tap % 2 == 0 else odd
|
||||||
|
target += source * (
|
||||||
|
0.5 * self.prototype[:, tap] * _tap_sign(tap))[None, None, :]
|
||||||
|
self.history[:] = joined[-(QMF_TAPS - 1):] if count else self.history
|
||||||
|
|
||||||
|
spectrum = (odd + 1j * even) * self.rotation[None, None, :]
|
||||||
|
transformed = np.fft.fft(spectrum, axis=-1)
|
||||||
|
output = np.empty(values.shape, dtype=np.complex128)
|
||||||
|
output[..., 0::2] = transformed[..., :QMF_BANDS // 2] / QMF_BANDS
|
||||||
|
output[..., 1::2] = np.conj(
|
||||||
|
transformed[..., QMF_BANDS // 2:][..., ::-1]) / QMF_BANDS
|
||||||
|
return output
|
||||||
|
|
||||||
|
|
||||||
|
class HybridAnalysis:
|
||||||
|
"""Sparse 64-QMF to 77-hybrid analysis: 11 taps, 3-slot high-band delay."""
|
||||||
|
|
||||||
|
def __init__(self, channels: int, kernel_data: str | Path = DEFAULT_CAPTURED_KERNEL_DATA):
|
||||||
|
if channels <= 0:
|
||||||
|
raise ValueError("channels must be positive")
|
||||||
|
tables = load_kernel_tables(kernel_data)
|
||||||
|
self.low_kernel = tables["hybrid"]
|
||||||
|
self.channels = int(channels)
|
||||||
|
self.history = np.zeros((HYBRID_TAPS - 1, self.channels, QMF_BANDS),
|
||||||
|
dtype=np.complex128)
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.history.fill(0.0)
|
||||||
|
|
||||||
|
def process_chunk(self, qmf) -> np.ndarray:
|
||||||
|
values = np.asarray(qmf, dtype=np.complex128)
|
||||||
|
if values.ndim != 3 or values.shape[1:] != (self.channels, QMF_BANDS):
|
||||||
|
raise ValueError(f"expected [slots,{self.channels},64], got {values.shape}")
|
||||||
|
count = values.shape[0]
|
||||||
|
joined = np.concatenate((self.history, values), axis=0)
|
||||||
|
output = np.zeros((count, self.channels, HYBRID_BANDS), dtype=np.complex128)
|
||||||
|
low = np.zeros((count, self.channels, HYBRID_LOW_BANDS), dtype=np.complex128)
|
||||||
|
for band, size in ((0, 8), (1, 4), (2, 4)):
|
||||||
|
offset = 0 if band == 0 else HYBRID_TAPS
|
||||||
|
spectrum = np.zeros((count, self.channels, size), dtype=np.complex128)
|
||||||
|
for tap in range(HYBRID_TAPS):
|
||||||
|
source = joined[tap:tap + count, :, band]
|
||||||
|
spectrum[:, :, (tap + 1) % size] += source * self.low_kernel[offset + tap]
|
||||||
|
base = 0 if band == 0 else 4 + band * 4
|
||||||
|
low[:, :, base:base + size] = np.fft.fft(spectrum, axis=-1)
|
||||||
|
output[:, :, :HYBRID_LOW_BANDS] = low[:, :, list(HYBRID_ORDER)]
|
||||||
|
for band in _CONJUGATED_BANDS:
|
||||||
|
output[:, :, band] = np.conj(output[:, :, band])
|
||||||
|
# The high bands pass through unchanged, delayed by HYBRID_DELAY slots.
|
||||||
|
delayed = joined[HYBRID_TAPS - 1 - HYBRID_DELAY:
|
||||||
|
HYBRID_TAPS - 1 - HYBRID_DELAY + count, :, 3:]
|
||||||
|
output[:, :, HYBRID_LOW_BANDS:] = delayed
|
||||||
|
self.history[:] = joined[-(HYBRID_TAPS - 1):] if count else self.history
|
||||||
|
return output
|
||||||
|
|
||||||
|
|
||||||
|
class HybridSynthesis:
|
||||||
|
"""Instantaneous 77-hybrid to 64-QMF recombination."""
|
||||||
|
|
||||||
|
def __init__(self, channels: int, kernel_data: str | Path = DEFAULT_CAPTURED_KERNEL_DATA):
|
||||||
|
if channels <= 0:
|
||||||
|
raise ValueError("channels must be positive")
|
||||||
|
self.channels = int(channels)
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
return None
|
||||||
|
|
||||||
|
def process_chunk(self, hybrid) -> np.ndarray:
|
||||||
|
values = np.asarray(hybrid, dtype=np.complex128)
|
||||||
|
if values.ndim != 3 or values.shape[1:] != (self.channels, HYBRID_BANDS):
|
||||||
|
raise ValueError(f"expected [slots,{self.channels},77], got {values.shape}")
|
||||||
|
output = np.empty((values.shape[0], self.channels, QMF_BANDS),
|
||||||
|
dtype=np.complex128)
|
||||||
|
output[..., 0] = (values[..., 0] + values[..., 2] + values[..., 4] +
|
||||||
|
values[..., 6] + values[..., 7] + values[..., 9] +
|
||||||
|
np.conj(values[..., 3]) + np.conj(values[..., 1]))
|
||||||
|
output[..., 1] = (values[..., 12] + values[..., 5] + values[..., 8] +
|
||||||
|
values[..., 10])
|
||||||
|
output[..., 2] = (values[..., 13] + values[..., 14] + values[..., 15] +
|
||||||
|
values[..., 11])
|
||||||
|
output[..., 3:] = values[..., HYBRID_LOW_BANDS:]
|
||||||
|
return output
|
||||||
|
|
||||||
|
|
||||||
|
class QmfSynthesis:
|
||||||
|
"""64-band synthesis: prototype-matched 10-tap overlap with a 9-slot history."""
|
||||||
|
|
||||||
|
def __init__(self, channels: int, kernel_data: str | Path = DEFAULT_CAPTURED_KERNEL_DATA):
|
||||||
|
if channels <= 0:
|
||||||
|
raise ValueError("channels must be positive")
|
||||||
|
tables = load_kernel_tables(kernel_data)
|
||||||
|
self.prototype = tables["qmf_prototype"]
|
||||||
|
self.rotation = tables["rotation"]
|
||||||
|
self.channels = int(channels)
|
||||||
|
self.history = np.zeros((QMF_TAPS - 1, self.channels, QMF_BANDS),
|
||||||
|
dtype=np.complex128)
|
||||||
|
|
||||||
|
def reset(self):
|
||||||
|
self.history.fill(0.0)
|
||||||
|
|
||||||
|
def process_chunk(self, qmf) -> np.ndarray:
|
||||||
|
values = np.asarray(qmf, dtype=np.complex128)
|
||||||
|
if values.ndim != 3 or values.shape[1:] != (self.channels, QMF_BANDS):
|
||||||
|
raise ValueError(f"expected [slots,{self.channels},64], got {values.shape}")
|
||||||
|
count = values.shape[0]
|
||||||
|
spectrum = np.empty((count, self.channels, QMF_BANDS), dtype=np.complex128)
|
||||||
|
spectrum[..., :QMF_BANDS // 2] = np.conj(values[..., 0::2])
|
||||||
|
spectrum[..., QMF_BANDS // 2:] = values[..., QMF_BANDS - 1::-2]
|
||||||
|
current = np.fft.fft(spectrum, axis=-1) * self.rotation[None, None, :]
|
||||||
|
joined = np.concatenate((self.history, current), axis=0)
|
||||||
|
output = np.zeros((count, self.channels, QMF_BANDS), dtype=np.float64)
|
||||||
|
reversed_bands = QMF_BANDS - 1 - np.arange(QMF_BANDS)
|
||||||
|
for lag in range(QMF_TAPS):
|
||||||
|
source = joined[QMF_TAPS - 1 - lag:QMF_TAPS - 1 - lag + count]
|
||||||
|
coefficient = (self.prototype[reversed_bands, QMF_TAPS - 1 - lag] *
|
||||||
|
_tap_sign(lag) * 2.0)
|
||||||
|
component = source.real if lag % 2 == 0 else source.imag
|
||||||
|
output += component * coefficient[None, None, :]
|
||||||
|
self.history[:] = joined[-(QMF_TAPS - 1):] if count else self.history
|
||||||
|
return output
|
||||||
Reference in new issue
Block a user