Read warp mode from OAMD instead of always scaling the floor Y coordinate.
For eligible layouts, mode 1 uses clamp(2Y, 0, 1), while mode 0 leaves Y
unchanged. Height coordinates are not scaled. Modes 2/3 remain explicitly
unsupported, and the CLI provides no warp override.
Keep separate coordinates for floor panning, height panning and trim
evaluation, with W denoting warp and B denoting front-back balance:
floor Y = B(W(Y))
height Y = B(Y)
trim Y = B(Y)
Apply explicit trims as a position-dependent gain for each object before
accumulation, replacing fixed gains applied to output channels after mixing.
Automatic, disabled and explicit trim modes are mutually exclusive.
Per-object bypass disables balance and trim while retaining warp.
Update controls at OAMD metadata events and retain the previous controls
when a payload has no trim element. Keep source trajectories and binaural
rendering unchanged.
Write the OAMD warp code into dbmd segment 9 while preserving the other
packed fields. Update the bilingual mathematical notes, README files and
code comments to describe parameters and rendering behavior.
Checked: warp 0/1, Y clipping, height coordinates, balance/trim ordering,
per-object bypass, metadata updates, bed positions and LFE passthrough.
332 lines
18 KiB
Markdown
332 lines
18 KiB
Markdown
# JustOneCacophony — JOC
|
||
|
||
[中文版](README.md)
|
||
|
||
> JustOneCacophony implements E-AC-3 JOC parsing, object reconstruction and rendering.
|
||
|
||
The project can extract and parse EMDF, ID14 JOC parameters, and ID11 OAMD metadata from common E-AC-3 JOC streams. It combines those data with the core PCM decoded by FFmpeg (N=5 or 7 full-band channels), reconstructs LFE plus 15 object channels, and writes ADM BWF, a WAV file for a selected speaker layout, or binaural stereo (a Rosella personalized model, or a compiled `.jochrtf`).
|
||
|
||
It covers the currently implemented stream forms. Unknown variants fail explicitly.
|
||
|
||
## Current features
|
||
|
||
- Scan common contiguous EMDF containers in E-AC-3 sync frames.
|
||
- Parse ID14 dense / sparse JOC parameters, Huffman data, differential matrices, and `joc_clipgain`.
|
||
- Parse ID11 OAMD position updates and build object trajectories.
|
||
- Read the bed layout the OAMD declares (5.1, 7.1, 7.1.2, 7.1.4, 9.1.4, 9.1.6, ...): for
|
||
bed-only content the 15 object slots are the bed channels, each rendered as a point
|
||
source at the fixed position its channel label defines (ETSI TS 103 420 table B.10);
|
||
speaker and binaural output share those source positions and never re-place them per
|
||
layout, while LFE keeps its separate path.
|
||
- Reconstruct LFE plus 15 object channels through analysis QMF, parameter interpolation, the object matrix, and inverse QMF.
|
||
- Write a 25-channel ADM BWF: a 10-channel 7.1.2 bed (silent except for LFE) plus 15 objects.
|
||
- Render directly to `2.0`, `3.1`, `4.0`, `5.1`, `7.1`, `5.1.2`, `5.1.4`, `7.1.2`, `7.1.4`, `9.1.4`
|
||
or `9.1.6` (`4.0` is the quad layout, a front and a back pair; see
|
||
[Speaker layouts and channel order](#speaker-layouts-and-channel-order)).
|
||
- Run binaural rendering directly from `pcm16 + ID11/OAMD`, without a temporary ADM BWF.
|
||
- Keep binaural DSP in float64/complex128: the personalized route retains its existing filterbank and latency rules, the `.jochrtf` route keeps its 77-band field and cross-frame state.
|
||
- Use a shared float32/PCM24 WAV writer and explicit PCM24 clipping policy for direct outputs.
|
||
- Implement everything in pure Python on NumPy;
|
||
- Read or write metadata sidecars and produce metadata, timing, and output reports.
|
||
|
||
## Processing flow
|
||
|
||
```text
|
||
M4A / E-AC-3
|
||
├─ FFmpeg extracts E-AC-3 and decodes the core PCM (N=5 or 7)
|
||
├─ EMDF → ID14 JOC parameters → object matrix
|
||
├─ core PCM → analysis QMF → parameter interpolation → inverse QMF
|
||
├─ ID11 OAMD → object positions and timing
|
||
└─ LFE + 15 objects
|
||
├─ 25ch ADM BWF
|
||
├─ speaker WAV for the selected layout
|
||
└─ direct ID11 timeline + HRTF (Rosella / .jochrtf) → binaural WAV
|
||
```
|
||
|
||
JOC object reconstruction, speaker rendering, and binaural rendering follow the same mathematics: analysis QMF, parameter interpolation, the object matrix, and inverse QMF. Bitstream parsing, the OAMD timeline, and CLI behavior are in Python as well.
|
||
|
||
## Requirements
|
||
|
||
- Python 3.10+
|
||
- NumPy 1.24+
|
||
- h5py 3.8+
|
||
- SciPy 1.10+
|
||
- A standalone FFmpeg executable; `ffmpeg-python` is not required. FFmpeg is discovered through `PATH` by default or selected with `--ffmpeg`. On startup the decoder options are probed with `ffmpeg -h decoder=eac3`: a missing E-AC-3 decoder or `-drc_scale` is a hard error, while a missing `-target_level` only fails when `--eac3-target-level` is used
|
||
|
||
Install the Python dependency in a project-specific environment:
|
||
|
||
```powershell
|
||
python -m pip install -r requirements.txt
|
||
```
|
||
|
||
If FFmpeg is not on `PATH`:
|
||
|
||
```powershell
|
||
python main.py input.m4a --ffmpeg C:\path\to\ffmpeg.exe
|
||
```
|
||
|
||
## Usage
|
||
|
||
Write a 25-channel ADM BWF by default:
|
||
|
||
```powershell
|
||
python main.py input.m4a
|
||
```
|
||
|
||
Select a backend or output path:
|
||
|
||
```powershell
|
||
python main.py input.eac3 -o output.adm.wav
|
||
```
|
||
|
||
Write a speaker-layout WAV directly:
|
||
|
||
```powershell
|
||
python main.py input.m4a --speaker-layout 2.0 --speaker-format float32
|
||
python main.py input.m4a --speaker-layout 5.1 --speaker-format int24
|
||
python main.py input.m4a --speaker-layout 7.1.2 --speaker-output output.7.1.2.wav
|
||
```
|
||
|
||
Write binaural stereo directly. Two kinds of input are available:
|
||
|
||
```powershell
|
||
# 1) Rosella .personalized_headphone (default; HRTF/binaural.personalized_headphone)
|
||
python main.py input.m4a --binaural
|
||
python main.py input.m4a --binaural --personalized-headphone C:\HRTF\subject.personalized_headphone
|
||
python main.py input.m4a --binaural --personalized-headphone --binaural-mode far --rosella-filterbank 2
|
||
|
||
# 2) .jochrtf compiled field (reserved for high-precision offline rendering)
|
||
python main.py input.m4a --binaural --compiled-hrtf-cache C:\HRTF\subject.jochrtf
|
||
python main.py input.m4a --binaural --compiled-hrtf-cache C:\HRTF\subject.jochrtf --jochrtf-room off
|
||
python main.py input.m4a --binaural --compiled-hrtf-cache C:\HRTF\subject.jochrtf --jochrtf-room on
|
||
python main.py input.m4a --binaural --binaural-output output.binaural.wav
|
||
```
|
||
|
||
- `.personalized_headphone` is a model produced by Dolby's official
|
||
personalization scan; its JSON parsing is implemented by this project
|
||
(`src/rosella_model.py`) and does not invoke any Dolby software. Near/Mid/Far
|
||
and the room handling are unchanged.
|
||
- `.jochrtf` is a directional field exported offline from a SOFA. This repository
|
||
loads and renders it; the compiler is a separate offline tool and its output
|
||
loads through `--compiled-hrtf-cache`.
|
||
|
||
The `.jochrtf` route keeps Near/Mid/Far point-source scales of approximately
|
||
1, 2.193 and 6.402 m, default Mid. It renders only the field's direct binaural
|
||
sound by default; `--jochrtf-room on` adds the project room (six first-order image
|
||
reflections plus an FDN tail, computed in code with no asset files).
|
||
input will retain its recorded room and ignore the additional room switch.
|
||
See [Binaural rendering](docs/binaural.en.md).
|
||
Because a cache contains transformed
|
||
HRTF data, its use and redistribution remain subject to the source dataset's
|
||
terms. See [Binaural Rendering](docs/binaural.en.md) and
|
||
[Third-party notices](THIRD_PARTY_NOTICES.md) for format boundaries, formulas,
|
||
state, timing, and distribution considerations.
|
||
|
||
Speaker and binaural output share peak analysis, the WAV writer, and clipping policy. When PCM24 may clip in a non-interactive environment, select a policy explicitly:
|
||
|
||
```powershell
|
||
python main.py input.m4a --speaker-layout 5.1 --speaker-format int24 --clip-action abort
|
||
python main.py input.m4a --speaker-layout 5.1 --speaker-format int24 --clip-action float32
|
||
python main.py input.m4a --speaker-layout 5.1 --speaker-format int24 --clip-action continue
|
||
python main.py input.m4a --binaural --compiled-hrtf-cache C:\HRTF\subject.jochrtf `
|
||
--binaural-format int24 --clip-action abort
|
||
```
|
||
|
||
Metadata and diagnostics:
|
||
|
||
```powershell
|
||
python main.py input.m4a --print-metadata summary
|
||
python main.py input.m4a --metadata-only --print-metadata frames
|
||
python main.py input.m4a --metadata-cache metadata_cache
|
||
python main.py input.m4a --metadata-dir metadata_cache
|
||
```
|
||
|
||
### E-AC-3 decode-side dynamic range and level
|
||
|
||
By default FFmpeg applies the stream `dynrng` dynamic range compression when
|
||
decoding E-AC-3 (`-drc_scale 1`). The core 5.1 PCM is the input of JOC object
|
||
reconstruction, and `dynrng` is playback-time gain, so it is inherited linearly
|
||
by every object and every output (ADM, speaker, binaural). This tool therefore
|
||
decodes at **full dynamic range** by default:
|
||
|
||
```powershell
|
||
python main.py input.m4a # default: -drc_scale 0, full range
|
||
python main.py input.m4a --eac3-drc-scale 1 # reproduce consumer playback
|
||
python main.py input.m4a --eac3-drc-scale 0.5 # apply half of it
|
||
python main.py input.m4a --eac3-target-level -27 # dialnorm-referenced level
|
||
```
|
||
|
||
- `--eac3-drc-scale` (`0`–`6`, default `0`) maps to FFmpeg `-drc_scale`: the gain
|
||
of each E-AC-3 block is `dynrng factor ^ value`. `0` disables DRC, `1` is the
|
||
author's intent, and `>1` is asymmetric (loud parts fully compressed, quiet
|
||
parts enhanced).
|
||
- `--eac3-target-level` (`-31`–`0`, default `0` = off) maps to FFmpeg
|
||
`-target_level`: a static per-frame gain of about `target_level - dialnorm` dB,
|
||
independent of and stackable with `--eac3-drc-scale`. dialnorm is a per-stream
|
||
property, so the attenuation that `-27` applies varies from stream to stream.
|
||
- The level change is expected: whether the gain is applied depends on the stream
|
||
`dynrng`, so `output_clip.peak` and the PCM24 clipping decision in
|
||
`.report.json` change accordingly.
|
||
- `--gain-db` is a static gain applied **after** reconstruction (float64 on the
|
||
binaural path) and is not the same thing as decode-side DRC, which is
|
||
block-varying; do not use `--gain-db` to cancel it.
|
||
- The `ffmpeg` field of `.report.json` records the FFmpeg version and the decode
|
||
options that were actually passed (`version`, `eac3_decode_options`).
|
||
|
||
### Binaural render mode
|
||
|
||
`--binaural-mode off|near|mid|far` selects the binaural render mode; the default
|
||
is `mid`, and both outputs share this single option:
|
||
|
||
- **Direct binaural rendering** (`--binaural`): `off` is rejected (error);
|
||
near/mid/far apply, defaulting to `mid`;
|
||
- **ADM BWF**: the low 3 binaural-render-mode bits of the last 15 JOC object
|
||
entries in DBMD segment 10 carry `off=0/near=1/far=2/mid=3`, leaving the first
|
||
10 bed entries unchanged; the default is `mid`, and `off` explicitly disables
|
||
the binaural metadata hint.
|
||
|
||
```powershell
|
||
python main.py input.m4a --binaural-mode mid
|
||
python main.py input.m4a --binaural-mode off # ADM BWF only: disable the DBMD hint
|
||
```
|
||
|
||
**The default `mid` is a human-specified rendering hint**; it is not original
|
||
binaural metadata extracted or recovered from the input E-AC-3 JOC bitstream,
|
||
nor does it represent the original mix's per-object binaural settings. The hint
|
||
does not change PCM, object trajectories, or direct speaker rendering. The
|
||
adjacent `.report.json` records `binaural_mode` (the mode name) and
|
||
`binaural_mode_value` (the ADM code; `null` for direct binaural output).
|
||
|
||
### OAMD time alignment
|
||
|
||
Object trajectories and direct speaker rendering both default to a metadata delay of `1473 samples`. This value describes the theoretical mapping between decoder-output PCM and OAMD updates. The speaker renderer retains its existing 32-sample control block, so the default update lands on effective block boundary `1472`:
|
||
|
||
```text
|
||
align32(1473) = 1472
|
||
```
|
||
|
||
Override the two paths with `--object-delay-samples` and `--speaker-metadata-offset`, respectively. The 1473-sample timing offset is distinct from the 640-value inverse-QMF filter/window state; 640 is a QMF state length, not a metadata delay.
|
||
|
||
The direct binaural path uses `--object-delay-samples`. Each ID11/OAMD event is
|
||
placed on an absolute sample timeline from its frame start, outer-subpayload
|
||
offset, and block offset, then shifted by that delay. Each 1536-sample input
|
||
frame is processed as three consecutive 512-sample blocks; the interpolated
|
||
position, direction, and profile are updated at each block's absolute starting
|
||
sample.
|
||
|
||
### Binaural calculation
|
||
|
||
See [Binaural Rendering Mathematics](docs/binaural.en.md) for QMF, hybrid processing, direction fields, distance, ITD, room processing, the 512-sample parameter updates above, and the selected filterbank's latency compensation (961 by default, 448 with `--rosella-filterbank 2`).
|
||
|
||
### Rosella filterbank
|
||
|
||
`--rosella-filterbank 1|2` selects the filterbank of the `.personalized_headphone`
|
||
path:
|
||
|
||
- `1` (default): the public bank, 13-tap hybrid / 6-slot high-band delay,
|
||
961 samples of latency;
|
||
- `2`: the 11-tap Rosella bank, 11-tap hybrid /
|
||
3-slot high-band delay, 448 samples of latency.
|
||
|
||
The option only applies to the Rosella model; the SOFA and `.jochrtf` paths
|
||
reject it. See [Rosella filterbank](docs/rosella_filterbank.en.md) for the
|
||
parameters, buffers and latency of both banks.
|
||
|
||
For all options:
|
||
|
||
```powershell
|
||
python main.py --help
|
||
```
|
||
|
||
Without `-o`, output goes to `output/` at the repository root.
|
||
|
||
## Speaker layouts and channel order
|
||
|
||
`--speaker-layout` accepts the eleven layouts below, and the PCM/WAV channel order is the order in
|
||
the table:
|
||
|
||
| Layout | Channel order |
|
||
|---|---|
|
||
| 2.0 | L, R |
|
||
| 3.1 | L, R, C, LFE |
|
||
| 4.0 | L, R, Lrs, Rrs |
|
||
| 5.1 | L, R, C, LFE, Ls, Rs |
|
||
| 7.1 | L, R, C, LFE, Ls, Rs, Lrs, Rrs |
|
||
| 5.1.2 | L, R, C, LFE, Ls, Rs, Ltm, Rtm |
|
||
| 5.1.4 | L, R, C, LFE, Ls, Rs, Ltf, Rtf, Ltr, Rtr |
|
||
| 7.1.2 | L, R, C, LFE, Ls, Rs, Lrs, Rrs, Ltm, Rtm |
|
||
| 7.1.4 | L, R, C, LFE, Ls, Rs, Lrs, Rrs, Ltf, Rtf, Ltr, Rtr |
|
||
| 9.1.4 | L, R, C, LFE, Ls, Rs, Lrs, Rrs, Lw, Rw, Ltf, Rtf, Ltr, Rtr |
|
||
| 9.1.6 | L, R, C, LFE, Ls, Rs, Lrs, Rrs, Lw, Rw, Ltf, Rtf, Ltm, Rtm, Ltr, Rtr |
|
||
|
||
`4.0` is the quad layout this project adds (the traditional `FL FR BL BR`, here `L R Lrs Rrs`); it is
|
||
not part of the Dolby layout set. Every other layout and channel order follows the Dolby
|
||
specifications.
|
||
|
||
Multichannel WAV files use `WAVE_FORMAT_EXTENSIBLE` with `dwChannelMask = 0`: the file declares no
|
||
Windows speaker positions, and the PCM channel order is exactly the table above, never rearranged
|
||
into mask-bit order. Players, ASIO, DAWs and other devices have to map the channel numbers
|
||
themselves; this project guarantees the content and numbering of each PCM channel, not that a
|
||
downstream player recognises Dolby speaker positions on its own.
|
||
|
||
The command line accepts only the eleven names in the table; anything else is rejected as an unknown
|
||
layout.
|
||
|
||
## Repository layout
|
||
|
||
```text
|
||
JustOneCacophony/
|
||
├─ main.py command-line entry point
|
||
├─ src/ Python implementation modules
|
||
├─ data/ Python runtime table data
|
||
├─ HRTF/ user HRTF data directory (create as needed, git-ignored)
|
||
├─ output/ output directory (create as needed; the .jochrtf cache defaults to its hrtf-cache subdirectory)
|
||
├─ docs/ math and binaural notes in both languages
|
||
├─ requirements.txt Python dependency
|
||
├─ README.md Chinese documentation
|
||
└─ README.en.md English documentation
|
||
```
|
||
|
||
## Mathematical implementation
|
||
|
||
The main documented stages are:
|
||
|
||
- dense JOC differential reconstruction and dequantization;
|
||
- parameter-band mapping to 64 QMF subbands;
|
||
- cross-frame parameter interpolation;
|
||
- analysis/inverse QMF, surround delay, and FIR state;
|
||
- the 1217-sample LFE delay;
|
||
- OAMD Q15 coordinate conversion;
|
||
- equal-power panning over target-layout regions;
|
||
- layout-dependent position compensation and sample-wise gain ramps;
|
||
- float32 and PCM24 output quantization;
|
||
- SOFA canonical import, full-HRIR directional interpolation, explicit 64-QMF/77-hybrid compact export, `36×2×77` fifth-order fields, exactly-once delay/phase, and special LFE.
|
||
|
||
See the [mathematical notes](docs/math.en.md) for the equations used by the decoding and rendering process.
|
||
|
||
## Known limitations
|
||
|
||
- Only the common contiguous EMDF transport is covered. Fragmented transport across multiple audio-block skip fields is not covered.
|
||
- All `joc_dmx_config_idx` values 0..4 are covered ($N=5,7,7,5,7$; the phase switch $\sigma(k)$ gates the Ls/Rs $-j$ rotation and the band-0 FIR together). The 7-channel configurations need an 8-channel bed and a dependent-substream `chanmap` to identify the extra pair: a missing independent substream or a missing `chanmap` is still a variant error, while a `chanmap` that disagrees with $k$ about the extra pair, or an OAMD bed mask that does, is recorded as a diagnostic and the frame still renders. The extra pair's **identity** is metadata only and never reaches the DSP (section 10.1); the bed mapping follows the channel codes the `chanmap` declares.
|
||
- When one audio frame holds several EMDF containers carrying ID11/ID14 (one of them may be something other than the frame's JOC parameters: its ID14 may declare more objects than its bytes can hold, and its ID11 may carry no group id), the container is chosen by the association rule of ETSI TS 103 420 section 8.2 and its Table 56: **the container whose OAMD and JOC payloads share a group id and whose ID14 parses completely** wins, then one whose ID14 merely parses; only a unique best score is used, the others are recorded as a diagnostic (`emdf/extra_container_ignored`), and a tie or all-zero score stays a variant error.
|
||
- The extra pair's identity follows TS 103 420 Table 47/Table 48 ($k=1\to$Lb/Rb, $k\in\{2,4\}\to$Tfl/Tfr, $N=5,7,7,5,7$); Annex E.3 requires a dependent-substream `chanmap`, when present, to indicate one of the Table 47 configurations, so `*_5_1_2top_*` (chanmap says Lts/Rts) does not satisfy that "shall". Such material is tolerated with a diagnostic instead of being refused.
|
||
- An audio frame with no JOC EMDF at all (for example when the encoder wrote none for the first frame) emits silent object channels and is recorded in the `[diagnostics]` line and the JSON report's `diagnostics` field.
|
||
- The speaker and SOFA binaural paths currently cover ordinary point objects; extent, spread, diffuse, divergence, channel lock, and similar controls are outside the supported scope.
|
||
- Speaker rendering reads warp 0/1, trims and front/back balance from OAMD, with no CLI warp override. Modes 2/3 are explicitly unsupported. Source trajectories and binaural remain free of these speaker controls; see [mathematics §12](docs/math.en.md) for parameters and evaluation order.
|
||
- Multi-data-point streams, uncommon band configurations, and unusual OAMD scheduling have less coverage than common 12-band, single-data-point material.
|
||
- `joc_clipgain` is folded into the **analysis-side scalar** (`scalar = 1/16 x g`, with `g` from the
|
||
`x:3 | y:5` field), so the x16 at the output cancels the 1/16 and the clamp sits **after** the gain.
|
||
The scalar is kept per frame: the first 256 samples ramp linearly from the previous frame's value and
|
||
the rest stays constant. For `g = 1` the interpolation is the identity and
|
||
`clamp(x)·1 == clamp(x·1)`.
|
||
- A speaker limiter is outside the current primary formula.
|
||
- The SOFA importer currently supports the strict `SimpleFreeFieldHRIR` FIR subset; other SOFA conventions require explicit adapters.
|
||
- The binaural runtime is fixed at 48 kHz, fifth order, and one measurement-radius shell at a time.
|
||
- ADM output, speaker layouts, and binaural models still need broader interoperability checks across platforms and players.
|
||
|
||
## Documentation
|
||
|
||
- [Mathematical notes](docs/math.en.md) · [中文](docs/math.md)
|
||
- [Binaural rendering](docs/binaural.en.md) · [中文](docs/binaural.md)
|
||
- [Rosella filterbank](docs/rosella_filterbank.en.md) · [中文](docs/rosella_filterbank.md)
|