Author SHA1 Message Date
TheM14 fae8e09b59 Put the sky bed channels back on the room walls
The sky bed channels (TFL/TFR, TSL/TSR, TBL/TBR) take x = -1 / +1 again instead
of the 62-level values -0.516113 / 0.516113; Y stays 1 / 0 / -1 and Z stays 1,
and the LFE rows stay at the origin because those channels carry no direction.
The horizontal pan of a point source only varies between the ends of a speaker
row, and the top row of every output layout spans 0.241943 .. 0.758057, so the
wall value still sits inside the full-power zone of either end speaker: the
decoded speaker output is unchanged, while the ADM position attribute and the
binaural fixed directions follow the wall geometry again.

Checked: the bed reference table produces the expected Q15 triples, the nine
output layouts still match the layout tables, and the regression suite passes.
2026-11-18 19:50:42 +08:00
TheM14 76074a05ba Put the CBA bed height row on the 62-level grid and render PCM 2.0/5.1/7.1 binaurally
The height row of the bed reference coordinates sits at ADM 62-level steps
15/62 and 47/62, not at the room corners: Tfl/Tfr, Tsl/Tsr and Tbl/Tbr move
from x = -1 / 1 to -0.516113 / 0.516113, which puts their Q15 numerators at
7928 / 24840 so every bed channel lands on its reference position instead of
being snapped to the nearest neighbour. Lw/Rw keep Y = 21/31 (5285). The
ADM-to-Q15 conversion scales by 32768, the full scale the renderer divides
by, so half-step positions such as 16384 and 7928 are no longer one count
low.

5.1, 5.1.2 and 5.1.4 keep their single surround pair at the rear corners
(y = 32767) instead of y = 16384. That family has no side pair, so its Ls/Rs
is the rear pair, which is the same point the multichannel PCM virtual
sources use; 7.1.x and 9.1.x keep Ls/Rs at 0.5 and Lrs/Rrs at 1.

Add the multichannel PCM binaural path. src/multichannel_binaural.py maps the
2.0 / 5.1 / 7.1 WAV channel order to synthetic objects (LFE on slot 0, the
full-band channels on slots 1..N in ffmpeg order, the remaining slots
silent), resamples the input to the native 48 kHz and drives the Rosella or
.jochrtf adapter. FixedPositionTimeline gives those static directions the
positions_at interface the OAMD timeline exposes, and both adapters accept an
optional fixed_positions argument instead of deriving a timeline from bed
labels.

Checked: the fifteen bed labels convert to the reference Q15 triples, the nine
output layouts agree with the layout table region by region, bed routing and
the derived direct bed labels are unchanged for every layout, and the
regression suite passes.
2026-11-18 17:05:30 +08:00
TheM14 7de17a8576 Carry OAMD object render metadata into the ADM BWF
The ADM writer only exported object positions, so the size, gain and divergence
present in the E-AC-3 JOC metadata never reached the ADM BWF.

build_adm_tracks now keeps the per-object render state that frame_state()
already produces (gain, size, divergence, snap).  The state is carried across
frames without an OAMD payload and a snapshot is attached to every ADM
keyframe; inactive objects write gain 0 so their PCM is muted in the ADM.
Divergence follows the speaker renderer's per-payload behaviour: it is taken
from the current payload and is 0 when the extended element does not carry it,
rather than holding a stale value.

adm_serializer writes the metadata on each audioBlockFormat:
  - size -> width/height/depth (relative units 0..1, OAMD width/depth/height
    order; all three are written together);
  - gain -> gain gainUnit="linear" (only when it is not unity);
  - divergence -> objectDivergence positionRange (only when non-zero).
Defaults (point source, 0 dB, no divergence) omit the elements, so existing
outputs are unchanged.

Verified on a 30 s E-AC-3 JOC excerpt: PCM and the default axml stay
byte-identical to the previous writer, the ADM validator passes, and a
metadata-override ADM carries width/height/depth with unchanged PCM.  The
speaker controls tests still pass.
2026-11-12 02:00:12 +08:00
TheM14 5fe0b16c51 Hold a frame without JOC metadata in the middle of a stream
A frame without JOC metadata (status 8) used to select passthrough in every
state, the middle of a stream included.  Passthrough replaces the object render
with the delayed core planes, and the object synthesis restarts from silence on
the next frame, so every such frame produced a step at the frame boundary and a
second one when the metadata came back.

Status 8 now follows the state machine like the other invalid statuses: state 0
(no valid frame rendered yet, i.e. plain E-AC-3 or before the first JOC frame)
keeps the passthrough, state 3 (mid-stream) selects hold and reuses the previous
object matrices, so the bed and the objects stay continuous.  The compatibility
note in the renderer records the old and the new state machine and states that
the choice is a concession for non-standard streams, not a normative reading.
2026-11-10 15:32:45 +08:00
TheM14 bd522cd0c4 Conceal a truncated JOC payload as an invalid frame status
A real E-AC-3 JOC stream carried one ID14 payload that ends inside joc_data:
the header, clipgain, sequence counter and object parameters are complete,
only the per-object matrices are missing.  The bit reader used to raise
IndexError there, so the metadata scan and the frame loop stopped instead of
playing.

The overrun now raises a typed JocTruncatedPayload, and parse_joc classifies
it as frame status 7 after the 12-bit joc_header: the parsed fields are kept
and the renderer takes its existing invalid-frame path (hold the previous
matrices).  A payload too short for the header still raises, the container
score ignores status 7, and the metadata summary reports a frame-status
histogram.
2026-11-09 23:51:37 +08:00
TheM14 42a05d7722 Implement OAMD metadata rendering and runtime table generation
- Add oamd_params/oamd_math for OAMD metadata state, ISF matrix assembly,
  runtime size-table generation and bed routing.
- Apply per-block gain/inactive, zone/elevation, size, snap, divergence,
  position refinement, priority budget, ISF and bed distribution in the
  speaker renderer.
- Compute gain, size, priority, position, refinement and delta values from
  formulas; generate the size spread kernel from speaker-layout geometry.
- Derive bed labels, routing, reference positions and automatic attenuation
  from speaker layouts.
- Correct the surround positions of 5.1, 5.1.2 and 5.1.4 and add the 6.0
  speaker layout.
- Update README, data README and math notes for the metadata path and
  runtime table generation.
2026-11-08 14:01:07 +08:00
TheM14 17e52a531c Fix JOC passthrough layout and no-previous-state interpolation
Interpolate the second half of a two-data-point frame from dp0 to dp1 when
the object has no previous matrix; a single data point keeps the hard cut.

Rebuild the passthrough frame as the 8-plane core order L, R, C, LFE, Ls,
Rs, (Lrs, Rrs) written to output slots 0..7, all delayed by 1217 samples,
and keep that 8-plane history updated on every frame. Frames without JOC
metadata always take this passthrough path, and a core channel count
change clears the LFE ring while keeping the clipgain ramp state.

Reject unsupported downmix configurations with a structured variant error
in the metadata scan, and score ID14 payloads by the frame status instead
of accepting any lenient parse.

Update the math notes and README for the two-data-point interpolation
without a previous state and the 8-plane passthrough output.
2026-11-06 11:53:05 +08:00
TheM14 0820444f45 Fix JOC clipgain, offset_ts, frame status, and renderer state machine
Decode clipgain multiplicatively as 2^(x-4)*(1 + y/32), keep the raw
5-bit offset_ts values and reject frames whose offsets leave 0..23 or are
not strictly increasing, and read the EMDF sample offset as
variable_bits(11). Report parse errors through the frame status: 2
unsupported downmix configuration, 3 object count clamped, 4 parameter
band index, 5 invalid offset_ts, 6 trailing data after byte padding.
Correct the 15/9/5/3 subband-to-parameter-band rows.

Apply the clipgain ramp to the LFE samples before the 1217-sample delay,
stop fading objects in from zero when they have no previous matrix, and
keep disappeared objects published with an all-zero matrix until they
reappear.

Render each frame through the seq state machine: hold the previous
matrices while the decoder is unsynchronized, hard cut on seq 0 or a core
channel count change, and pass the delayed core PCM through when no JOC
metadata is present. Keep the 16-channel output as LFE + 15 objects so
CBA bed channels stay on the fixed-position object path that the ADM,
speaker, and binaural backends already consume.

Update the math notes and README for the multiplicative clipgain, the
no-previous-state interpolation, the LFE gain ramp, and the frame-level
decoding behavior. State that E-AC-3 input without JOC metadata is
outside the supported scope.
2026-11-06 11:35:59 +08:00
TheM14 4aef8db36e Implement PLIIx and Lo/Ro folding for warp modes 2/3
- Render 5.1-family and 4.0 layouts through expanded surround geometry before folding.
- Keep trim, balance and automatic position compensation tied to the final target layout.
- Preserve height channels, LFE passthrough and per-object trim bypass behavior.
- Update the bilingual README and mathematical documentation.
2026-11-03 13:02:41 +08:00
TheM14 0e28cd085c Honor OAMD warp modes and position-dependent speaker trims
Read warp mode from OAMD instead of always scaling the floor Y coordinate.
For eligible layouts, mode 1 uses clamp(2Y, 0, 1), while mode 0 leaves Y
unchanged. Height coordinates are not scaled. Modes 2/3 remain explicitly
unsupported, and the CLI provides no warp override.

Keep separate coordinates for floor panning, height panning and trim
evaluation, with W denoting warp and B denoting front-back balance:

    floor Y  = B(W(Y))
    height Y = B(Y)
    trim Y   = B(Y)

Apply explicit trims as a position-dependent gain for each object before
accumulation, replacing fixed gains applied to output channels after mixing.
Automatic, disabled and explicit trim modes are mutually exclusive.
Per-object bypass disables balance and trim while retaining warp.

Update controls at OAMD metadata events and retain the previous controls
when a payload has no trim element. Keep source trajectories and binaural
rendering unchanged.

Write the OAMD warp code into dbmd segment 9 while preserving the other
packed fields. Update the bilingual mathematical notes, README files and
code comments to describe parameters and rendering behavior.

Checked: warp 0/1, Y clipping, height coordinates, balance/trim ordering,
per-object bypass, metadata updates, bed positions and LFE passthrough.
2026-11-03 00:11:23 +08:00
TheM14 8424e6bdfb Apply the OAMD trim and front-back balance in speaker rendering
Select the trim configuration that applies to the output layout from its
surround and height speaker counts:

    cfg = kappa_f(surrounds) + 3 * kappa_h(heights)

Trim and balance apply to seven layouts: 2.0, 3.1, 4.0, 5.1, 5.1.2, 5.1.4
and 7.1.  Wider layouts are left untrimmed.

An explicit configuration contributes a static level for the centre,
surround and height groups, which multiplies the panning gains per output
channel.  Automatic and disabled configurations contribute no static
level: the attenuation an automatic configuration describes is already
applied per object by the layout position compensation.

An explicit configuration also contributes the two front-back balance
values.  Writing n = round(16 |b|) and s = n / 18, each balance
reparameterises the OAMD front-back axis in [0, 1]:

    y <= 0.05 or y >= 0.95        -> y
    y < 0.5,  b > 0               -> (1 + s) y - 0.05 s
    y >= 0.5, b > 0               -> (1 - s) (y - 0.5) + 0.5 + 0.45 s
    y < 0.5,  b < 0               -> (1 - s) y + 0.05 s
    y >= 0.5, b < 0               -> (1 + s) (y - 0.5) + 0.5 - 0.45 s

The map is monotonic onto [0.05, 0.95], keeps both walls fixed, and needs
no clamping.  Height blends the listener-plane and overhead values before
panning; the horizontal and vertical coordinates are untouched.

Warp is not read here: it selects a downmix mode and does not enter the
speaker path.
2026-11-02 17:18:53 +08:00
TheM14 220e791ce9 Carry the OAMD trim element into the ADM BWF metadata
Parse the trim element: warp_mode, global_trim_mode, the nine
per-configuration trim_balance_presence records, and the per-object
bypass flags.  Select the configuration that applies to a layout from
its surround and height speaker counts:

    cfg = kappa_f(surrounds) + 3 * kappa_h(heights)

    kappa_f: 0 -> 0, 2..3 -> 1, otherwise 2
    kappa_h: 0 -> 0, 2 -> 1, otherwise 2

Write the configuration to both carriers:

  * AXML gets a DolbyAtmosMasterFile container holding
    DolbyMasterContentInfo, DolbyTrimModeSetup (one records entry per
    configuration with centerTrim, surroundTrim, heightTrim and the two
    balance fields) and DolbyWarpMode.
  * Segment 10 of the dbmd chunk gets the nine 15-byte records: bit 0
    is the automatic flag, and an explicit record carries three float32
    trims followed by two sign-magnitude balance bytes.

Configurations left at default keep the automatic flag and carry no
value elements.
2026-11-02 17:00:24 +08:00
35 changed files with 4109 additions and 538 deletions

No files matched your search

+24 -20
View File
@@ -2,11 +2,11 @@
[中文版](README.md) [中文版](README.md)
> JustOneCacophony is an experimental/test implementation of E-AC-3 JOC for studying JOC parsing, reconstruction, rendering, and the associated mathematics. > JustOneCacophony implements E-AC-3 JOC parsing, object reconstruction and rendering.
The project can extract and parse EMDF, ID14 JOC parameters, and ID11 OAMD metadata from common E-AC-3 JOC streams. It combines those data with the core PCM decoded by FFmpeg (N=5 or 7 full-band channels), reconstructs LFE plus 15 object channels, and writes ADM BWF, a WAV file for a selected speaker layout, or binaural stereo (a Rosella personalized model, or a compiled `.jochrtf`). The project can extract and parse EMDF, ID14 JOC parameters, and ID11 OAMD metadata from common E-AC-3 JOC streams. It combines those data with the core PCM decoded by FFmpeg (N=5 or 7 full-band channels), reconstructs LFE plus 15 object channels, and writes ADM BWF, a WAV file for a selected speaker layout, or binaural stereo (a Rosella personalized model, or a compiled `.jochrtf`).
This is research code, not a complete, standards-compliant, or production-grade JOC decoder. It covers only the stream forms currently implemented. Unknown variants fail explicitly—because when the math goes wrong, all that may remain is the cacophony. It covers the currently implemented stream forms. Unknown variants fail explicitly.
## Current features ## Current features
@@ -15,12 +15,12 @@ This is research code, not a complete, standards-compliant, or production-grade
- Parse ID11 OAMD position updates and build object trajectories. - Parse ID11 OAMD position updates and build object trajectories.
- Read the bed layout the OAMD declares (5.1, 7.1, 7.1.2, 7.1.4, 9.1.4, 9.1.6, ...): for - Read the bed layout the OAMD declares (5.1, 7.1, 7.1.2, 7.1.4, 9.1.4, 9.1.6, ...): for
bed-only content the 15 object slots are the bed channels, each rendered as a point bed-only content the 15 object slots are the bed channels, each rendered as a point
source at the fixed position its channel label defines (ETSI TS 103 420 table B.10); source at the fixed position its channel label defines; speaker and binaural output
speaker and binaural output share those source positions and never re-place them per share those source positions and never re-place them per layout, while LFE keeps its
layout, while LFE keeps its separate path. separate path.
- Reconstruct LFE plus 15 object channels through analysis QMF, parameter interpolation, the object matrix, and inverse QMF. - Reconstruct LFE plus 15 object channels through analysis QMF, parameter interpolation, the object matrix, and inverse QMF.
- Write a 25-channel ADM BWF: a 10-channel 7.1.2 bed (silent except for LFE) plus 15 objects. - Write a 25-channel ADM BWF: a 10-channel 7.1.2 bed (silent except for LFE) plus 15 objects.
- Render directly to `2.0`, `3.1`, `4.0`, `5.1`, `7.1`, `5.1.2`, `5.1.4`, `7.1.2`, `7.1.4`, `9.1.4` - Render directly to `2.0`, `3.1`, `4.0`, `5.1`, `6.0`, `7.1`, `5.1.2`, `5.1.4`, `7.1.2`, `7.1.4`, `9.1.4`
or `9.1.6` (`4.0` is the quad layout, a front and a back pair; see or `9.1.6` (`4.0` is the quad layout, a front and a back pair; see
[Speaker layouts and channel order](#speaker-layouts-and-channel-order)). [Speaker layouts and channel order](#speaker-layouts-and-channel-order)).
- Run binaural rendering directly from `pcm16 + ID11/OAMD`, without a temporary ADM BWF. - Run binaural rendering directly from `pcm16 + ID11/OAMD`, without a temporary ADM BWF.
@@ -225,12 +225,12 @@ path:
- `1` (default): the public bank, 13-tap hybrid / 6-slot high-band delay, - `1` (default): the public bank, 13-tap hybrid / 6-slot high-band delay,
961 samples of latency; 961 samples of latency;
- `2`: the Rosella bank captured from the reference renderer, 11-tap hybrid / - `2`: the 11-tap Rosella bank, 11-tap hybrid /
3-slot high-band delay, 448 samples of latency. 3-slot high-band delay, 448 samples of latency.
The option only applies to the Rosella model; the SOFA and `.jochrtf` paths The option only applies to the Rosella model; the SOFA and `.jochrtf` paths
reject it. See [Rosella filterbank](docs/rosella_filterbank.en.md) for the reject it. See [Rosella filterbank](docs/rosella_filterbank.en.md) for the
parameters, buffers, latency and validation of both banks. parameters, buffers and latency of both banks.
For all options: For all options:
@@ -238,11 +238,11 @@ For all options:
python main.py --help python main.py --help
``` ```
Without `-o`, output still goes to `output/` at the repository root. The directory move intentionally preserves this behavior. Without `-o`, output goes to `output/` at the repository root.
## Speaker layouts and channel order ## Speaker layouts and channel order
`--speaker-layout` accepts the eleven layouts below, and the PCM/WAV channel order is the order in `--speaker-layout` accepts the twelve layouts below, and the PCM/WAV channel order is the order in
the table: the table:
| Layout | Channel order | | Layout | Channel order |
@@ -251,6 +251,7 @@ the table:
| 3.1 | L, R, C, LFE | | 3.1 | L, R, C, LFE |
| 4.0 | L, R, Lrs, Rrs | | 4.0 | L, R, Lrs, Rrs |
| 5.1 | L, R, C, LFE, Ls, Rs | | 5.1 | L, R, C, LFE, Ls, Rs |
| 6.0 | L, R, Ls, Rs, Lrs, Rrs |
| 7.1 | L, R, C, LFE, Ls, Rs, Lrs, Rrs | | 7.1 | L, R, C, LFE, Ls, Rs, Lrs, Rrs |
| 5.1.2 | L, R, C, LFE, Ls, Rs, Ltm, Rtm | | 5.1.2 | L, R, C, LFE, Ls, Rs, Ltm, Rtm |
| 5.1.4 | L, R, C, LFE, Ls, Rs, Ltf, Rtf, Ltr, Rtr | | 5.1.4 | L, R, C, LFE, Ls, Rs, Ltf, Rtf, Ltr, Rtr |
@@ -293,11 +294,13 @@ The main documented stages are:
- dense JOC differential reconstruction and dequantization; - dense JOC differential reconstruction and dequantization;
- parameter-band mapping to 64 QMF subbands; - parameter-band mapping to 64 QMF subbands;
- cross-frame parameter interpolation; - cross-frame parameter interpolation (hard cut when no previous state exists, zero-matrix publishing after an object disappears);
- the frame-level JOC state machine (normal, hold, hard cut, core passthrough);
- analysis/inverse QMF, surround delay, and FIR state; - analysis/inverse QMF, surround delay, and FIR state;
- the 1217-sample LFE delay; - the 1217-sample LFE delay with its sample-wise clipgain ramp;
- OAMD Q15 coordinate conversion; - OAMD Q15 coordinate conversion;
- equal-power panning over target-layout regions; - equal-power panning over target-layout regions;
- OAMD metadata application (per-block gain and inactive, zone/elevation, size kernel, snap, divergence, position refinement, priority size budget, ISF matrices, bed distribution);
- layout-dependent position compensation and sample-wise gain ramps; - layout-dependent position compensation and sample-wise gain ramps;
- float32 and PCM24 output quantization; - float32 and PCM24 output quantization;
- SOFA canonical import, full-HRIR directional interpolation, explicit 64-QMF/77-hybrid compact export, `36×2×77` fifth-order fields, exactly-once delay/phase, and special LFE. - SOFA canonical import, full-HRIR directional interpolation, explicit 64-QMF/77-hybrid compact export, `36×2×77` fifth-order fields, exactly-once delay/phase, and special LFE.
@@ -310,15 +313,16 @@ See the [mathematical notes](docs/math.en.md) for the equations used by the deco
- All `joc_dmx_config_idx` values 0..4 are covered ($N=5,7,7,5,7$; the phase switch $\sigma(k)$ gates the Ls/Rs $-j$ rotation and the band-0 FIR together). The 7-channel configurations need an 8-channel bed and a dependent-substream `chanmap` to identify the extra pair: a missing independent substream or a missing `chanmap` is still a variant error, while a `chanmap` that disagrees with $k$ about the extra pair, or an OAMD bed mask that does, is recorded as a diagnostic and the frame still renders. The extra pair's **identity** is metadata only and never reaches the DSP (section 10.1); the bed mapping follows the channel codes the `chanmap` declares. - All `joc_dmx_config_idx` values 0..4 are covered ($N=5,7,7,5,7$; the phase switch $\sigma(k)$ gates the Ls/Rs $-j$ rotation and the band-0 FIR together). The 7-channel configurations need an 8-channel bed and a dependent-substream `chanmap` to identify the extra pair: a missing independent substream or a missing `chanmap` is still a variant error, while a `chanmap` that disagrees with $k$ about the extra pair, or an OAMD bed mask that does, is recorded as a diagnostic and the frame still renders. The extra pair's **identity** is metadata only and never reaches the DSP (section 10.1); the bed mapping follows the channel codes the `chanmap` declares.
- When one audio frame holds several EMDF containers carrying ID11/ID14 (one of them may be something other than the frame's JOC parameters: its ID14 may declare more objects than its bytes can hold, and its ID11 may carry no group id), the container is chosen by the association rule of ETSI TS 103 420 section 8.2 and its Table 56: **the container whose OAMD and JOC payloads share a group id and whose ID14 parses completely** wins, then one whose ID14 merely parses; only a unique best score is used, the others are recorded as a diagnostic (`emdf/extra_container_ignored`), and a tie or all-zero score stays a variant error. - When one audio frame holds several EMDF containers carrying ID11/ID14 (one of them may be something other than the frame's JOC parameters: its ID14 may declare more objects than its bytes can hold, and its ID11 may carry no group id), the container is chosen by the association rule of ETSI TS 103 420 section 8.2 and its Table 56: **the container whose OAMD and JOC payloads share a group id and whose ID14 parses completely** wins, then one whose ID14 merely parses; only a unique best score is used, the others are recorded as a diagnostic (`emdf/extra_container_ignored`), and a tie or all-zero score stays a variant error.
- The extra pair's identity follows TS 103 420 Table 47/Table 48 ($k=1\to$Lb/Rb, $k\in\{2,4\}\to$Tfl/Tfr, $N=5,7,7,5,7$); Annex E.3 requires a dependent-substream `chanmap`, when present, to indicate one of the Table 47 configurations, so `*_5_1_2top_*` (chanmap says Lts/Rts) does not satisfy that "shall". Such material is tolerated with a diagnostic instead of being refused. - The extra pair's identity follows TS 103 420 Table 47/Table 48 ($k=1\to$Lb/Rb, $k\in\{2,4\}\to$Tfl/Tfr, $N=5,7,7,5,7$); Annex E.3 requires a dependent-substream `chanmap`, when present, to indicate one of the Table 47 configurations, so `*_5_1_2top_*` (chanmap says Lts/Rts) does not satisfy that "shall". Such material is tolerated with a diagnostic instead of being refused.
- An audio frame with no JOC EMDF at all (for example when the encoder wrote none for the first frame) emits silent object channels and is recorded in the `[diagnostics]` line and the JSON report's `diagnostics` field. - E-AC-3 input without JOC metadata (ID11 + ID14) is outside the supported scope. If an individual audio frame has no JOC EMDF, that frame passes the core PCM through (the 8 core planes are written to output slots 0..7 by identity, each delayed by 1217 samples) and records `joc_frame_missing` in `diagnostics`.
- The speaker and SOFA binaural paths currently cover ordinary point objects; extent, spread, diffuse, divergence, channel lock, and similar controls are outside the supported scope. - Speaker rendering applies the OAMD object attributes: per-block gain/inactive, zone/elevation, size/snap/divergence, the priority size budget, element 5 position refinement, ISF basis matrices, and bed distribution. The SOFA binaural path still covers ordinary point objects only; these object controls are outside the binaural path's scope.
- OAMD trim elements are boundary-checked and skipped; warp, balance, and trim parameters are not applied to raw object trajectories or speaker rendering. - Speaker rendering reads warp, trims and front/back balance from OAMD, with no CLI warp override. Generic rendering parameters support warp 0/1 and modes 2/3, which render to 7.1.x before PLIIx or Standard (Lo/Ro) folding to 5.1.x; calibration always follows the final target layout. Source trajectories and binaural do not bake in these speaker controls; see [mathematics §12](docs/math.en.md) for parameters and evaluation order.
- Multi-data-point streams, uncommon band configurations, and unusual OAMD scheduling have less coverage than common 12-band, single-data-point material. - Multi-data-point streams, uncommon band configurations, and unusual OAMD scheduling have less coverage than common 12-band, single-data-point material.
- `joc_clipgain` is folded into the **analysis-side scalar** (`scalar = 1/16 x g`, with `g` from the - `joc_clipgain` is folded into the **analysis-side scalar** (`scalar = 1/16 x g`, with
`x:3 | y:5` field), so the x16 at the output cancels the 1/16 and the clamp sits **after** the gain. `g = 2^(x-4)·(1 + y/32)` from the `x:3 | y:5` field, range 0.0625..15.75), so the x16 at the output
The scalar is kept per frame: the first 256 samples ramp linearly from the previous frame's value and cancels the 1/16 and the clamp sits **after** the gain. The scalar is kept per frame: the first 256
the rest stays constant. For `g = 1` the interpolation is the identity and samples ramp linearly from the previous frame's value and the rest stays constant. For `g = 1` the
`clamp(x)·1 == clamp(x·1)`. interpolation is the identity and `clamp(x)·1 == clamp(x·1)`. LFE bypasses the analysis but applies
the same full `g` as a sample-wise ramp before its 1217-sample delay.
- A speaker limiter is outside the current primary formula. - A speaker limiter is outside the current primary formula.
- The SOFA importer currently supports the strict `SimpleFreeFieldHRIR` FIR subset; other SOFA conventions require explicit adapters. - The SOFA importer currently supports the strict `SimpleFreeFieldHRIR` FIR subset; other SOFA conventions require explicit adapters.
- The binaural runtime is fixed at 48 kHz, fifth order, and one measurement-radius shell at a time. - The binaural runtime is fixed at 48 kHz, fifth order, and one measurement-radius shell at a time.
+19 -15
View File
@@ -2,11 +2,11 @@
[English](README.en.md) [English](README.en.md)
> JustOneCacophony 是一个 E-AC-3 JOC 的实验性 / 测试实现,用于研究 JOC 的解析、重建、渲染以及相关数学过程。 > JustOneCacophony 实现 E-AC-3 JOC 的解析、对象重建与渲染。
项目可以从常见 E-AC-3 JOC 码流中提取并解析 EMDF、ID14 JOC 参数和 ID11 OAMD 元数据,结合 FFmpeg 解码出的核心 PCM(N=5 或 7 个全频段通道)重建 LFE 与 15 路对象 PCM,并输出 ADM BWF、指定扬声器布局的 WAV,或输出双耳 WAV(Rosella 个性化模型,或已编译的 `.jochrtf`)。 项目可以从常见 E-AC-3 JOC 码流中提取并解析 EMDF、ID14 JOC 参数和 ID11 OAMD 元数据,结合 FFmpeg 解码出的核心 PCM(N=5 或 7 个全频段通道)重建 LFE 与 15 路对象 PCM,并输出 ADM BWF、指定扬声器布局的 WAV,或输出双耳 WAV(Rosella 个性化模型,或已编译的 `.jochrtf`)。
这是研究代码,不是完整、标准兼容或生产级的 JOC 解码器。它只覆盖当前已实现的码流形态;遇到未知变体时会明确报错,而不是假装一切都很和谐——如果哪里算错了,它可能就真的只剩 cacophony 了。 当前覆盖已实现的码流形态;遇到未知变体时明确报错。
## 当前功能 ## 当前功能
@@ -14,11 +14,11 @@
- 解析 ID14 dense / sparse JOC 参数、Huffman 数据、差分矩阵与 `joc_clipgain`; - 解析 ID14 dense / sparse JOC 参数、Huffman 数据、差分矩阵与 `joc_clipgain`;
- 解析 ID11 OAMD 位置更新并生成对象轨迹; - 解析 ID11 OAMD 位置更新并生成对象轨迹;
- 读取 OAMD 声明的 bed 布局(5.1、7.1、7.1.2、7.1.4、9.1.4、9.1.6 等):bed-only 内容的 - 读取 OAMD 声明的 bed 布局(5.1、7.1、7.1.2、7.1.4、9.1.4、9.1.6 等):bed-only 内容的
15 路对象就是 bed 声道,每个声道按其声道标签的固定方位(ETSI TS 103 420 Table B.10) 15 路对象就是 bed 声道,每个声道按其声道标签的固定方位作为点声源渲染,扬声器与双耳
作为点声源渲染,扬声器与双耳共用同一组源位置,不随输出布局改变;LFE 独立处理; 共用同一组源位置,不随输出布局改变;LFE 独立处理;
- 通过 analysis QMF、参数插值、对象矩阵和 inverse QMF 重建 LFE + 15 路对象 PCM; - 通过 analysis QMF、参数插值、对象矩阵和 inverse QMF 重建 LFE + 15 路对象 PCM;
- 输出 25 声道 ADM BWF:10 声道 7.1.2 bed(除 LFE 外静音)+ 15 个对象; - 输出 25 声道 ADM BWF:10 声道 7.1.2 bed(除 LFE 外静音)+ 15 个对象;
- 直接渲染 `2.0`、`3.1`、`4.0`、`5.1`、`7.1`、`5.1.2`、`5.1.4`、`7.1.2`、`7.1.4`、`9.1.4`、 - 直接渲染 `2.0`、`3.1`、`4.0`、`5.1`、`6.0`、`7.1`、`5.1.2`、`5.1.4`、`7.1.2`、`7.1.4`、`9.1.4`、
`9.1.6`(`4.0` 是 Quad 布局,前后各一对;声道排列见[扬声器布局与声道顺序](#扬声器布局与声道顺序)); `9.1.6`(`4.0` 是 Quad 布局,前后各一对;声道排列见[扬声器布局与声道顺序](#扬声器布局与声道顺序));
- 从 `pcm16 + ID11/OAMD` 直接运行双耳渲染,不生成临时 ADM BWF; - 从 `pcm16 + ID11/OAMD` 直接运行双耳渲染,不生成临时 ADM BWF;
- 双耳 DSP 全程使用 float64/complex128;个性化路线保留原滤波器组与延迟规则,`.jochrtf` 路线保留 77 带方向场与跨帧状态; - 双耳 DSP 全程使用 float64/complex128;个性化路线保留原滤波器组与延迟规则,`.jochrtf` 路线保留 77 带方向场与跨帧状态;
@@ -193,11 +193,11 @@ outer subpayload offset 与 block offset 落到绝对 sample timeline,再加
`--rosella-filterbank 1|2` 选择 `.personalized_headphone` 路径使用的滤波器组: `--rosella-filterbank 1|2` 选择 `.personalized_headphone` 路径使用的滤波器组:
- `1`(默认):公开滤波器组,13-tap hybrid / 6-slot 高频延迟,961 samples 延迟; - `1`(默认):公开滤波器组,13-tap hybrid / 6-slot 高频延迟,961 samples 延迟;
- `2`:从参考渲染器捕获的 Rosella 滤波器组,11-tap hybrid / 3-slot 高频延迟, - `2`:11-tap Rosella 滤波器组,11-tap hybrid / 3-slot 高频延迟,
448 samples 延迟。 448 samples 延迟。
该选项只作用于 Rosella 模型;SOFA 与 `.jochrtf` 路径给出时报错。两套实现的参数、 该选项只作用于 Rosella 模型;SOFA 与 `.jochrtf` 路径给出时报错。两套实现的参数、
缓冲、延迟与验证见 [Rosella 滤波器组](docs/rosella_filterbank.md)。 缓冲与延迟见 [Rosella 滤波器组](docs/rosella_filterbank.md)。
更多参数可查看: 更多参数可查看:
@@ -205,11 +205,11 @@ outer subpayload offset 与 block offset 落到绝对 sample timeline,再加
python main.py --help python main.py --help
``` ```
未指定 `-o` 时,输出仍写入仓库根目录的 `output/`。这是文件移动后特意保持的原有行为。 未指定 `-o` 时,输出写入仓库根目录的 `output/`。
## 扬声器布局与声道顺序 ## 扬声器布局与声道顺序
`--speaker-layout` 只接受下表 11 个布局,PCM/WAV 的声道顺序就是表里的顺序: `--speaker-layout` 只接受下表 12 个布局,PCM/WAV 的声道顺序就是表里的顺序:
| 布局 | 声道顺序 | | 布局 | 声道顺序 |
|---|---| |---|---|
@@ -217,6 +217,7 @@ python main.py --help
| 3.1 | L, R, C, LFE | | 3.1 | L, R, C, LFE |
| 4.0 | L, R, Lrs, Rrs | | 4.0 | L, R, Lrs, Rrs |
| 5.1 | L, R, C, LFE, Ls, Rs | | 5.1 | L, R, C, LFE, Ls, Rs |
| 6.0 | L, R, Ls, Rs, Lrs, Rrs |
| 7.1 | L, R, C, LFE, Ls, Rs, Lrs, Rrs | | 7.1 | L, R, C, LFE, Ls, Rs, Lrs, Rrs |
| 5.1.2 | L, R, C, LFE, Ls, Rs, Ltm, Rtm | | 5.1.2 | L, R, C, LFE, Ls, Rs, Ltm, Rtm |
| 5.1.4 | L, R, C, LFE, Ls, Rs, Ltf, Rtf, Ltr, Rtr | | 5.1.4 | L, R, C, LFE, Ls, Rs, Ltf, Rtf, Ltr, Rtr |
@@ -255,11 +256,13 @@ JustOneCacophony/
- dense JOC 差分还原与去量化; - dense JOC 差分还原与去量化;
- 参数带到 64 个 QMF 子带的映射; - 参数带到 64 个 QMF 子带的映射;
- 跨帧参数插值; - 跨帧参数插值(无上一帧时硬切,对象消失后按零矩阵发布);
- 帧级 JOC 状态机(正常 / 保持 / 硬切 / 核心直通);
- analysis / inverse QMF、环绕声道延迟与 FIR 状态; - analysis / inverse QMF、环绕声道延迟与 FIR 状态;
- LFE 1217-sample 延迟; - LFE 1217-sample 延迟与逐点 clipgain 斜坡;
- OAMD Q15 坐标转换; - OAMD Q15 坐标转换;
- 基于目标布局 region 的等功率声像; - 基于目标布局 region 的等功率声像;
- OAMD 元数据应用(每块增益与 inactive、zone/elevation、size 核、snap、divergence、位置细化、priority 尺寸预算、ISF 矩阵、bed 分配);
- 布局位置补偿与逐样本增益斜坡; - 布局位置补偿与逐样本增益斜坡;
- float32 与 PCM24 输出量化; - float32 与 PCM24 输出量化;
- SOFA canonical importer、完整HRIR方向插值、显式64-QMF/77-hybrid压缩导出、`36×2×77`五阶方向field、exactly-once delay/phase与special LFE。 - SOFA canonical importer、完整HRIR方向插值、显式64-QMF/77-hybrid压缩导出、`36×2×77`五阶方向field、exactly-once delay/phase与special LFE。
@@ -272,13 +275,14 @@ JustOneCacophony/
- 覆盖 `joc_dmx_config_idx` 0..4 的全部档位($N=5,7,7,5,7$,相位开关 $\sigma(k)$ 同时门控 Ls/Rs 的 $-j$ 旋转与 band-0 FIR)。7 通道档位必须解出 8 通道床,且需要依赖子流的 `chanmap` 才能确定额外那一对:缺少核心子流或缺少 `chanmap` 仍按变体报错;`chanmap` 声明的额外对与 $k$ 不一致、或 OAMD bed 掩码与 $k$ 不一致时,只记入 `diagnostics` 并继续解码——额外那对的**身份**只进元数据、不进 DSP(§10.1),床映射按 `chanmap` 的通道码做。 - 覆盖 `joc_dmx_config_idx` 0..4 的全部档位($N=5,7,7,5,7$,相位开关 $\sigma(k)$ 同时门控 Ls/Rs 的 $-j$ 旋转与 band-0 FIR)。7 通道档位必须解出 8 通道床,且需要依赖子流的 `chanmap` 才能确定额外那一对:缺少核心子流或缺少 `chanmap` 仍按变体报错;`chanmap` 声明的额外对与 $k$ 不一致、或 OAMD bed 掩码与 $k$ 不一致时,只记入 `diagnostics` 并继续解码——额外那对的**身份**只进元数据、不进 DSP(§10.1),床映射按 `chanmap` 的通道码做。
- 同一音频帧里出现多个含 ID11/ID14 的 EMDF 容器时(其中一个容器可能不是本帧的 JOC 参数:ID14 声明的对象数超出其字节所能容纳的范围,ID11 也可能没有 groupid),按 ETSI TS 103 420 §8.2 + Table 56 的关联规则打分选取:**OAMD 与 JOC 载荷 groupid 相同且 ID14 能完整解析**的容器优先,其次只看 ID14 能否完整解析;只有唯一最高分才采用,其余记入 `diagnostics`(`emdf/extra_container_ignored`),并列或全部为 0 仍报变体。 - 同一音频帧里出现多个含 ID11/ID14 的 EMDF 容器时(其中一个容器可能不是本帧的 JOC 参数:ID14 声明的对象数超出其字节所能容纳的范围,ID11 也可能没有 groupid),按 ETSI TS 103 420 §8.2 + Table 56 的关联规则打分选取:**OAMD 与 JOC 载荷 groupid 相同且 ID14 能完整解析**的容器优先,其次只看 ID14 能否完整解析;只有唯一最高分才采用,其余记入 `diagnostics`(`emdf/extra_container_ignored`),并列或全部为 0 仍报变体。
- 额外那一对的身份以 TS 103 420 Table 47/Table 48 为准($k=1\to$Lb/Rb,$k\in\{2,4\}\to$Tfl/Tfr,$N=5,7,7,5,7$);Annex E.3 规定依赖子流的 `chanmap`(若存在)**应**指明 Table 47 的某种配置,因此 `*_5_1_2top_*`(chanmap 写 Lts/Rts)属于不满足 "shall" 的素材,我们按"记诊断 + 继续解码"容错处理。 - 额外那一对的身份以 TS 103 420 Table 47/Table 48 为准($k=1\to$Lb/Rb,$k\in\{2,4\}\to$Tfl/Tfr,$N=5,7,7,5,7$);Annex E.3 规定依赖子流的 `chanmap`(若存在)**应**指明 Table 47 的某种配置,因此 `*_5_1_2top_*`(chanmap 写 Lts/Rts)属于不满足 "shall" 的素材,我们按"记诊断 + 继续解码"容错处理。
- 音频帧内没有任何 JOC EMDF 时(例如编码器未写首帧元数据),该帧的对象通道静音,并在 stderr 的 `[diagnostics]` 与 JSON 报告的 `diagnostics` 里记一笔。 - 不含 JOC 元数据(ID11 + ID14)的 E-AC-3 输入不在支持范围;解码过程中若个别音频帧没有 JOC EMDF,该帧输出核心 PCM 直通(8 路核心按 identity 写入输出槽 0..7,每路延迟 1217 采样),并以 `joc_frame_missing` 记入 `diagnostics`。
- 扬声器与 SOFA 双耳路径当前只覆盖普通点对象;extent、spread、diffuse、divergence、channel lock 等对象控制不在支持范围内。 - 扬声器渲染应用 OAMD 的对象属性:每块增益/inactive、zone/elevation、size/snap/divergence、priority 尺寸预算、element 5 位置细化、ISF 基矩阵和 bed distribute;SOFA 双耳路径仍只覆盖普通点对象,这些对象控制不在双耳路径范围内。
- OAMD trim element 会按声明边界校验并跳过;warp、balance 和 trim 参数不应用于当前原始对象轨迹或扬声器渲染。 - 扬声器渲染从 OAMD 元数据读取 warp、trim 和 front/back balance;命令行不提供 warp 覆盖。通用渲染参数支持 warp 0/1,以及先渲染到 7.1.x 再按 PLIIx 或 Standard(Lo/Ro)折回 5.1.x 的 warp 2/3;校准参数始终按最终目标布局选择。原始对象轨迹与 binaural 不预处理这些扬声器控制,参数和顺序见 [数学说明 §12](docs/math.md)。
- 多数据点、少见参数带配置和特殊 OAMD 调度的覆盖度低于常见 12-band、单数据点素材。 - 多数据点、少见参数带配置和特殊 OAMD 调度的覆盖度低于常见 12-band、单数据点素材。
- `joc_clipgain` 在**分析端**并入分析标量:`标量 = 1/16 × g`(`g = 1 + (y/32)·2^(x−4)`),出口的 - `joc_clipgain` 在**分析端**并入分析标量:`标量 = 1/16 × g`(`g = 2^(x−4)·(1 + y/32)`,值域 0.0625..15.75),出口的
×16 抵消 1/16,净效果是 `×g`,clamp 因此处在增益**之后**。标量按帧保存两个槽(上一帧值 → 本帧值), ×16 抵消 1/16,净效果是 `×g`,clamp 因此处在增益**之后**。标量按帧保存两个槽(上一帧值 → 本帧值),
本帧前 256 个采样线性过渡、之后为常数,复位后的第一帧直接取新值;`g = 1` 时插值恒等,且 `clamp(x)·1 == clamp(x·1)`。 本帧前 256 个采样线性过渡、之后为常数,复位后的第一帧直接取新值;`g = 1` 时插值恒等,且 `clamp(x)·1 == clamp(x·1)`。
LFE 走独立通路,但使用同一个完整 `g` 的逐点斜坡,并乘在 1217-sample 延迟之前。
- 扬声器 limiter 不属于当前实现的主公式。 - 扬声器 limiter 不属于当前实现的主公式。
- SOFA importer 当前严格支持 `SimpleFreeFieldHRIR` FIR;其它 SOFA convention 需要显式 adapter。 - SOFA importer 当前严格支持 `SimpleFreeFieldHRIR` FIR;其它 SOFA convention 需要显式 adapter。
- 双耳 runtime 固定 48 kHz、五阶和一次选择一个 measurement-radius shell。 - 双耳 runtime 固定 48 kHz、五阶和一次选择一个 measurement-radius shell。
+19 -8
View File
@@ -19,6 +19,21 @@ joc_huff_code_7ch_pos_index_sparse int64[6,2]
`src/joc_qmf.py` loads the QMF tables, while `src/joc_decode.py` loads the JOC Huffman trees. `src/joc_qmf.py` loads the QMF tables, while `src/joc_decode.py` loads the JOC Huffman trees.
## ISF coefficient data
`oamd/metadata_params.json` provides the ISF compressed base coefficients and
layout descriptors as lookup data. Gain, size, priority, position, divergence,
delta and refinement values are computed in code from formulas or fixed
constants; the size spread kernel is generated at runtime from the
`speaker_layouts` region geometry with grouped equal-power panning, an S448
weight and a power sum (20/20/8 virtual points, 20 size nodes, 35/4 position
nodes). Bed labels, routing and reference positions are derived from
`speaker_layouts` as well.
`src/oamd_params.py` parses the data file lazily and `src/oamd_math.py`
computes coefficients in a fixed float32 order; `src/speaker_renderer.py`
applies them to the object target coefficients.
## Binaural rendering tables ## Binaural rendering tables
`rosella_kernels.npz` contains the fixed 64-QMF/77-hybrid tables used by `rosella_kernels.npz` contains the fixed 64-QMF/77-hybrid tables used by
@@ -72,10 +87,9 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q
The same values also appear in other public implementations of these standards The same values also appear in other public implementations of these standards
(for example FFmpeg's `aacps_tablegen.h` and `aacsbrdata.h`). (for example FFmpeg's `aacps_tablegen.h` and `aacsbrdata.h`).
### Captured Rosella filterbank ### 11-tap Rosella filterbank
`rosella_captured_kernels.npz` holds the bank the reference headphone renderer `rosella_captured_kernels.npz` holds the bank selected by `--rosella-filterbank 2`:
actually runs (`--rosella-filterbank 2`):
```text ```text
format_version little-endian int32[1] format_version little-endian int32[1]
@@ -85,11 +99,8 @@ hybrid complex64[22]
``` ```
The table values are promoted to float64/complex128 on load. The full The table values are promoted to float64/complex128 on load. The full
analysis/synthesis latency is 448 samples (961 for the public bank). These analysis/synthesis latency is 448 samples (961 for the public bank). This file
values come from runtime buffer captures of the reference renderer, not from the contains no personalized HRTF or room tuning.
public standards above; the C++ core compiles the same data in as
`src/hrtf/rosella_filterbank_tables.h`. This file also contains no personalized
HRTF or room tuning.
Public availability of a standard does not by itself grant permission to Public availability of a standard does not by itself grant permission to
practice related patent claims. practice related patent claims.
+14 -6
View File
@@ -19,6 +19,17 @@ joc_huff_code_7ch_pos_index_sparse int64[6,2]
`src/joc_qmf.py` 读取 QMF 表,`src/joc_decode.py` 读取 JOC Huffman 树。 `src/joc_qmf.py` 读取 QMF 表,`src/joc_decode.py` 读取 JOC Huffman 树。
## ISF 系数数据
`oamd/metadata_params.json` 提供 ISF 压缩基系数与布局描述的查表数据。
gain、size、priority、位置、divergence 与差分/细化表在代码中按公式或
固定常量计算;size 扩散核按分组等功率声像、S448 权重和幂和,在运行时从
`speaker_layouts` 的区域几何生成(20/20/8 虚拟点、20 个尺寸节点、35/4 个
位置节点);bed 的标签、路由和参考位置也由 `speaker_layouts` 推导。
`src/oamd_params.py` 延迟解析数据文件,`src/oamd_math.py` 按固定的 float32 运算
顺序生成系数;`src/speaker_renderer.py` 把它们接到对象目标系数上。
## 双耳渲染表 ## 双耳渲染表
`rosella_kernels.npz` 保存`.jochrtf`压缩渲染和个性化bank 1使用的64-QMF/77-hybrid固定表: `rosella_kernels.npz` 保存`.jochrtf`压缩渲染和个性化bank 1使用的64-QMF/77-hybrid固定表:
@@ -64,10 +75,9 @@ $$Y_p = \sum_{q\in C_p}\Bigl(\operatorname{Re}X_q + j\,s_q\,\operatorname{Im}X_q
相同数值可在 FFmpeg(`aacps_tablegen.h`、`aacsbrdata.h`)等公开实现中查到。 相同数值可在 FFmpeg(`aacps_tablegen.h`、`aacsbrdata.h`)等公开实现中查到。
### Rosella 捕获版滤波器组 ### 11-tap Rosella 滤波器组
`rosella_captured_kernels.npz` 保存参考头戴渲染器实际运行的那套滤波器组 `rosella_captured_kernels.npz` 保存 `--rosella-filterbank 2` 使用的滤波器组:
(`--rosella-filterbank 2`):
```text ```text
format_version little-endian int32[1] format_version little-endian int32[1]
@@ -77,9 +87,7 @@ hybrid complex64[22]
``` ```
表值载入后提升为 float64/complex128;全链 latency 为 448 samples(公开版为 961)。 表值载入后提升为 float64/complex128;全链 latency 为 448 samples(公开版为 961)。
这些数值来自参考渲染器的运行时缓冲捕获,不是上述公开标准的推导值;C++ 内核以 文件不含个性化 HRTF 或 room 调参。
`src/hrtf/rosella_filterbank_tables.h` 的形式内置同一组数据。文件同样不含个性化
HRTF 或 room 调参。
标准可公开获取不等于获准实施相关专利。 标准可公开获取不等于获准实施相关专利。
+1
View File
@@ -0,0 +1 @@
{"isf":{"configs":[{"idx":0,"counts":[3,1,0,0,4],"primary_row_map":[0,1,2,3],"secondary_row_map":[0,2,1,3],"matrix_row_stride":73,"matrix_rows":4,"matrix_values":[0.8065544962882996,0.5517973303794861,0.6069754362106323,0.3953828811645508,0.5534031987190247,0.5539230704307556,0.24255745112895966,0.47505271434783936,0.444728285074234,0.17205633223056793,0.4247773289680481,0.5103204250335693,0.18820077180862427,0.17929865419864655,0.452311247587204,-0.280463844537735,0.017338519915938377,-0.28909528255462646,0.0,0.0560874417424202,-0.2892860770225525,-0.013385031372308731,0.10039377957582474,-0.016056396067142487,-0.2825709581375122,-0.04064895957708359,0.10803624987602234,0.05630350857973099,-0.047425154596567154,-0.26719823479652405,-0.06350012123584747,0.097655288875103,0.1266908049583435,-0.029927069321274757,-0.04877259582281113,-0.2535346746444702,-0.050417669117450714,0.09264937043190002,0.07186891883611679,0.05187152326107025,-0.027603687718510628,-0.044422321021556854,-0.2613919973373413,0.0,0.03838589787483215,-0.12560494244098663,0.051136989146471024,-0.09426405280828476,0.06614412367343903,-0.05140332132577896,-0.1441955715417862,0.08474995940923691,-0.036616962403059006,-0.11979598551988602,0.07123114913702011,-0.03924524784088135,-0.03968926891684532,-0.13764509558677673,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.4201984107494354,-0.056425247341394424,0.4477916657924652,0.01371981855481863,0.44707077741622925,-0.05664261803030968,0.05117806792259216,0.4387761950492859,0.02798357792198658,0.05597246438264847,0.4283769726753235,-0.05218393728137016,0.02862025611102581,0.049770887941122055,0.4346124231815338,0.9679709672927856,0.18938815593719482,0.8870320916175842,0.0,0.5337828993797302,0.6927342414855957,-0.048569101840257645,0.36494335532188416,0.4001503586769104,0.6377595067024231,-0.11768897622823715,0.21134333312511444,0.5358392596244812,0.258344829082489,0.5548434853553772,-0.16074039041996002,0.14455674588680267,0.40664923191070557,0.4540003538131714,0.18882706761360168,0.4997166693210602,-0.1372985541820526,0.15293832123279572,0.16712871193885803,0.4936601221561432,0.19720278680324554,0.19437994062900543,0.5300741195678711,0.0,-0.04083092510700226,0.1691703349351883,-0.05588478967547417,0.1370640993118286,-0.07459205389022827,0.08213182538747787,0.1942090392112732,-0.09941107779741287,0.06275865435600281,0.17987869679927826,-0.08116155862808228,0.064299076795578,0.0623021274805069,0.1853865683078766,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,-0.4335499405860901,-0.056425247341394424,-0.3872392475605011,-0.08972250670194626,-0.36937597393989563,-0.05664261803030968,-0.08691781014204025,-0.3389882743358612,-0.11113959550857544,-0.07409896701574326,-0.3167206048965454,-0.05218393728137016,-0.05974094942212105,-0.07198946177959442,-0.32919612526893616,0.09978409111499786,-0.08105161786079407,0.14544576406478882,0.0,-0.15092352032661438,0.2103460729122162,0.025524256750941277,-0.14595693349838257,-0.06471376121044159,0.22086507081985474,0.06565067917108536,-0.11256188154220581,-0.15150493383407593,-0.004101958591490984,0.22987015545368195,0.09314629435539246,-0.08828219771385193,-0.17176778614521027,-0.06250102072954178,0.013875355944037437,0.2309974730014801,0.07791588455438614,-0.08850760757923126,-0.08191754668951035,-0.13957907259464264,-0.012519026175141335,0.007122464943677187,0.23086495697498322,0.0,-0.046477995812892914,0.1691703349351883,-0.06098373979330063,0.11757846176624298,-0.0774788111448288,0.05829664319753647,0.1942090392112732,-0.09704835712909698,0.03865424543619156,0.14466993510723114,-0.08293674886226654,0.043386269360780716,0.045827239751815796,0.1853865683078766,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.006118591409176588,0.0,0.006118591409176588,0.0,0.006118591409176588,0.0,0.0,0.006118591409176588,0.0,0.0,0.006118591409176588,0.0,0.0,0.0,0.006118591409176588,0.6063833832740784,0.3690401017665863,0.3751586675643921,0.0,0.611859142780304,0.0,0.0,0.0,0.611859142780304,0.0,0.0,0.0,0.611859142780304,0.0,0.0,0.0,0.0,0.0,0.611859142780304,0.0,0.0,0.0,0.0,0.0,0.0,0.611859142780304,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.0,0.6125019788742065,0.3751586675643921,0.3751586675643921,0.0,0.6125019788742065,0.0,0.0,0.3751586675643921,0.3751586675643921,0.0,0.0,0.0,0.6125019788742065,0.0,0.0]},{"idx":1,"counts":[5,3,0,0,8],"primary_row_map":[0,1,2,3,4,5,6,7],"secondary_row_map":[0,4,3,2,1,5,7,6],"matrix_row_stride":73,"matrix_rows":8,"matrix_values":[0.7620719075202942,0.878515899181366,0.37703391909599304,0.6023024916648865,0.30184605717658997,0.7353588342666626,0.3631182610988617,0.22095273435115814,0.5886307954788208,0.23342837393283844,0.17362885177135468,0.666203498840332,0.2959349751472473,0.23725862801074982,0.19081860780715942,-0.04823773726820946,-0.14693273603916168,-0.07366316765546799,0.0,-0.23600386083126068,0.035208798944950104,0.020941175520420074,-0.15896914899349213,-0.17113232612609863,Line truncated
+283 -18
View File
@@ -147,15 +147,17 @@ If the clipgain field consists of integer $x$ and mantissa $y$, then
$$ $$
G_{\mathrm{clip}}= G_{\mathrm{clip}}=
1+\frac{y}{32}2^{x-4}. 2^{x-4}\left(1+\frac{y}{32}\right).
$$ $$
It is folded into the analysis-side scalar $S=\frac{1}{16}G_{\mathrm{clip}}$, which multiplies Its range is $[0.0625,15.75]$; at $x=4$ it reduces to $1+y/32$. It is folded into the analysis-side
every core-bed sample before the QMF analysis; the $\times16$ after the object inverse QMF cancels the scalar $S=\frac{1}{16}G_{\mathrm{clip}}$, which multiplies every core-bed sample before the QMF
$1/16$, so the net effect is a multiply by $G_{\mathrm{clip}}$ and the $[-1,1]$ clamp sits **after** the analysis; the $\times16$ after the object inverse QMF cancels the $1/16$, so the net effect is a multiply
gain. The scalar keeps two per-frame slots: the first 256 samples of a frame ramp linearly from the by $G_{\mathrm{clip}}$ and the $[-1,1]$ clamp sits **after** the gain. The scalar keeps two per-frame
previous frame's value to this frame's value and the rest stays constant, and the first frame after a slots: the first 256 samples of a frame ramp linearly from the previous frame's value to this frame's
reset takes the new value directly. LFE bypasses this scalar. value and the rest stays constant, and the first frame after a reset takes the new value directly. LFE
uses a separate path but applies the same full $G_{\mathrm{clip}}$ as a sample-wise ramp before its
1217-sample delay; see section 8.
## 3. Parameter-band expansion and time interpolation ## 3. Parameter-band expansion and time interpolation
@@ -190,7 +192,13 @@ Thus $p(b)=k$ if and only if $b\in\mathcal B_k$. Other parameter-band counts use
### 3.2 One-data-point interpolation ### 3.2 One-data-point interpolation
Let $P_{o,c,b}$ be the previous frame-end value and $D_{o,c,p(b)}$ the current target. For slot $t=0\ldots23$: Let $P_{o,c,b}$ be the previous frame-end value and $D_{o,c,p(b)}$ the current target. When the object
has no previous state yet (the first frame of the stream, or the object's first appearance), a single
data point keeps all 24 slots at $D_{o,c,p(b)}$ (hard cut, no fade-in from zero); with two data points,
the first 12 slots stay at $D_0$ and the last 12 slots still interpolate from $D_0$ to $D_1$ using the
second segment of section 3.3 (with $P$ set to $D_0$).
With a previous state, for slot $t=0\ldots23$:
$$ $$
\alpha_t=\frac{t+1}{24}, \alpha_t=\frac{t+1}{24},
@@ -208,11 +216,31 @@ $$
M_{o,c,b,23}=D_{o,c,p(b)}. M_{o,c,b,23}=D_{o,c,p(b)}.
$$ $$
This value then becomes the previous state for the next frame. This value then becomes the previous state for the next frame. Once an object's present flag goes from 1
to 0, it keeps being published with an all-zero matrix and its previous state is updated to zero until it
becomes present again.
### 3.3 Multiple data points ### 3.3 Multiple data points
When a frame contains two data points, `offset_ts` gives the segment boundary. Each segment uses the same linear relation between the previous and next targets; step mode switches targets at the designated slot. `offset_ts` gives the first slot at which each data point becomes active, using the raw 5-bit value
(legal range $0\ldots23$); the values of one object must be strictly increasing. In the default mode
(`slope_idx=0`), two data points $D_0$ and $D_1$ interpolate as
$$
M_{o,c,b,t}=
\begin{cases}
(1-\alpha_t)P_{o,c,b}+\alpha_tD_{0}, & t=0\ldots10,\ \alpha_t=\dfrac{t+1}{12},\\[1mm]
D_{0}, & t=11,\\[1mm]
(1-\beta_t)D_{0}+\beta_tD_{1}, & t=12\ldots23,\ \beta_t=\dfrac{t-11}{12}.
\end{cases}
$$
When no previous state exists, the same formulas apply with $P_{o,c,b}$ set to $D_0$.
Step mode (`slope_idx=1`) switches targets directly at the designated slots: for $t<o_0$ it keeps the
previous frame-end value $P_{o,c,b}$ (or $D_0$ when no previous state exists), for $o_0\le t<o_1$ it
uses $D_0$, and for $t\ge o_1$ it uses $D_1$; with a single data point it keeps $P$ for $t<o_0$ and uses
$D_0$ from $o_0$ on.
## 4. Analysis QMF for core PCM ## 4. Analysis QMF for core PCM
@@ -337,15 +365,21 @@ where $r=0\ldots63$. Synthesis state must advance continuously by slot.
## 8. LFE path ## 8. LFE path
LFE bypasses the object matrix and inverse QMF and uses a 1217-sample delay. After the input and output scale factors cancel: LFE bypasses the object matrix and inverse QMF. The core LFE samples are first multiplied by the
sample-wise gain ramp $G[n]$ (the same full $G_{\mathrm{clip}}$ as section 2.4: for $n<256$,
$G[n]=G_{\mathrm{old}}+n(G_{\mathrm{new}}-G_{\mathrm{old}})/256$; for $n\ge256$, $G[n]=G_{\mathrm{new}}$;
on the first frame or after a reset, $G_{\mathrm{old}}=G_{\mathrm{new}}$). The scaled samples then
enter the 1217-sample delay, and the read-out is only clamped, with no further gain:
$$ $$
y_{\mathrm{LFE}}[n]= y_{\mathrm{LFE}}[n]=
\mathrm{clip}\left( \mathrm{clip}\left(
x_{\mathrm{LFE,core}}[n-1217],-1,1 x_{\mathrm{LFE,core}}[n-1217]\cdot G[n-1217],-1,1
\right). \right).
$$ $$
Here $G[k]$ uses the frame-local position of input sample $k$, i.e. $k\bmod1536$.
## 9. OAMD coordinates ## 9. OAMD coordinates
The lateral and longitudinal grids use $N=62$; the height grid uses $N=15$. The quantizer is The lateral and longitudinal grids use $N=62$; the height grid uses $N=15$. The quantizer is
@@ -358,7 +392,7 @@ q_N(k)=
\right). \right).
$$ $$
OAR coordinates are The normalized room coordinates are
$$ $$
u=\frac{q_1}{32768}, u=\frac{q_1}{32768},
@@ -446,14 +480,16 @@ $$
$$ $$
For layouts with one horizontal surround pair rather than separate side and rear pairs (the 5.1 For layouts with one horizontal surround pair rather than separate side and rear pairs (the 5.1
family, and 4.0, which carries one rear pair only), the longitudinal coordinate is family, and 4.0, which carries one rear pair only), OAMD `warp_mode=1` uses
$$ $$
v_{\mathrm{floor}}= v_{\mathrm{floor}}=
\mathrm{clamp}(2v,0,1). \mathrm{clamp}(2v,0,1).
$$ $$
Other layouts use $v_{\mathrm{floor}}=v$. Warp 0 and other layouts use $v_{\mathrm{floor}}=v$. The height layer is not scaled.
An absent warp indication uses 0; no CLI override is provided. Modes 2/3 use
the intermediate layout and folding matrix in §12.5. Balance follows this scaling.
### 10.3 Height layer ### 10.3 Height layer
@@ -474,6 +510,9 @@ $$
## 11. Layout-dependent position compensation ## 11. Layout-dependent position compensation
This is the per-object gain for automatic trim. Disabled trim uses unity;
explicit trim uses §12.2 instead, without multiplying automatic compensation again.
Let $N_h$ be the number of relevant height speakers and $N_f$ the number of relevant additional horizontal speakers: Let $N_h$ be the number of relevant height speakers and $N_f$ the number of relevant additional horizontal speakers:
$$ $$
@@ -519,7 +558,188 @@ G_{\mathrm{pos}}
\mathbf G_{\mathrm{point}}. \mathbf G_{\mathrm{point}}.
$$ $$
## 12. OAMD time alignment and gain ramps ## 12. OAMD trim and front-back balance
### 12.1 Configuration selection
The OAMD trim element splits a programme's calibration data into nine
configurations, chosen by the target layout's additional horizontal speaker
count $N_f$ and height speaker count $N_h$:
$$
\kappa_f(N)=
\begin{cases}
0, & N=0,\\
1, & N\in\{2,3\},\\
2, & \text{otherwise},
\end{cases}
\qquad
\kappa_h(N)=
\begin{cases}
0, & N=0,\\
1, & N=2,\\
2, & \text{otherwise}.
\end{cases}
$$
$$
c_{\mathrm{trim}}=\kappa_f(N_f)+3\,\kappa_h(N_h).
$$
The nine configurations enumerate "no / some / many additional horizontal
speakers by no / some / many height speakers".
Trim and front-back balance apply to seven layouts only: 2.0, 3.1, 4.0, 5.1,
5.1.2, 5.1.4 and 7.1. Wider layouts (7.1.2, 7.1.4, 9.1.4, 9.1.6) are left
untrimmed, with $g_t=1$ and $b=0$.
Warp, trim and balance controls are read from OAMD metadata:
| Parameter | Value or unit | Behavior |
|---|---|---|
| `warp_mode` | 0 / 1 / 2 / 3 | Direct rendering / floor Y scaling / PLIIx folding / Standard (Lo/Ro) folding |
| `global_trim_mode` | 0 / 1 / 2 | Automatic / disabled / per-layout configuration; reserved value 3 is rejected |
| $T_s$, $T_h$ | dB | Explicit surround and height trim, with positive values treated as zero |
| $T_c$ | dB | Signed explicit centre trim |
| $b_{\mathrm{lis}}$, $b_{\mathrm{tb}}$ | $[-1,1]$, step $1/16$ | Listener-plane and overhead front-back balance |
| Per-object trim bypass | Boolean | Disables the object's balance and trim gain while retaining warp |
With global mode 2, the selected configuration's default or disabled flag
chooses automatic or disabled trim. A missing configuration uses automatic;
otherwise its explicit parameters apply. Automatic uses §11, disabled uses
unity, and explicit uses §12.2. These modes are mutually exclusive.
Without an initial trim element, warp is 0 and trim is automatic. A later
payload without that element retains the previous controls; a new element
updates them at the corresponding metadata event boundary. The CLI has no
warp override.
### 12.2 Per-object levels
Centre, surround and height levels parameterize a position-dependent object
gain; they are not fixed gains applied to the output channels after mixing.
For original Y after balance $y_t$ and $z=\max(w,0)$:
$$
a_s=\mathrm{clamp}(2y_t,0,1)(1-z),\quad a_h=z,
$$
$$
a_c=1-\mathrm{clamp}\bigl(10\max(|z|-0.05,y_t-0.05,|u-0.5|-0.05),0,1\bigr).
$$
Explicit trim uses
$$
D=a_s\min(T_s,0)+a_h\min(T_h,0)+a_cT_c,\qquad g_t=10^{D/20}.
$$
The scalar multiplies the object's whole panning vector before accumulation.
Automatic trim uses §11; disabled trim or per-object bypass uses unity. LFE
passes through. Gain evaluation, panning interpolation and object accumulation
use float64.
### 12.3 Front-back balance
Each configuration also carries two front-back balance values:
$b_{\mathrm{lis}}$ for the listener plane and $b_{\mathrm{tb}}$ for the overhead
layer. Both are quantised to a $1/16$ grid and are non-zero only in an explicit
configuration.
Writing $n=\mathrm{round}(16\,|b|)$ and $s=n/18$, the single-layer map is
$$
B_b(y)=
\begin{cases}
y, & y\le 0.05\ \text{or}\ y\ge 0.95,\\[2pt]
(1+s)y-0.05s, & 0.05<y<0.5,\ b>0,\\[2pt]
(1-s)(y-0.5)+0.5+0.45s, & 0.5\le y<0.95,\ b>0,\\[2pt]
(1-s)y+0.05s, & 0.05<y<0.5,\ b<0,\\[2pt]
(1+s)(y-0.5)+0.5-0.45s, & 0.5\le y<0.95,\ b<0.
\end{cases}
$$
It maps $[0.05,0.95]$ monotonically onto itself: the two walls stay fixed, the
two halves meet at $y=0.5$, and their slopes are $1+s$ and $1-s$. The 5 percent
wall bands and the case $b=0$ pass through unchanged, so the map never leaves
the interval and needs no extra clamp.
The height $w$ blends the two layers:
$$
y'=(1-w)\,B_{b_{\mathrm{lis}}}(y)+w\,B_{b_{\mathrm{tb}}}(y).
$$
The front-back axis $y$ is oriented as in §9, with $0$ at the front wall and
$1$ at the back wall; $x$ and $z$ are unchanged.
### 12.4 Where it is applied
Let $B_z(y)$ denote the balance map of §12.3 and $W(y)$ the warp map of §10.2.
Direct point rendering in warp modes 0/1 keeps three coordinates:
$$
y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y).
$$
Floor and height panning use $y_f,y_h$; trim evaluation uses $y_t$. The final gain is
$$
g_{o,c}=G_{\mathrm{object}}\,g_t(u,y_t,z)\,P_c(u,y_f,y_h,z).
$$
Warp precedes balance, followed by trim/pan gain composition, but not every
stage shares a warped Y. Balance requires explicit trim without object bypass;
warp remains independent of trim state and bypass. Source positions are kept
unchanged for ADM and binaural.
Floor coordinates pass through warp and then balance before floor panning.
Height and trim coordinates pass through balance alone. The trim scalar
multiplies the object's entire panning vector before accumulation into output
channels. LFE passes through.
### 12.5 Surround folding in warp modes 2/3
For a 5.1, 5.1.2 or 5.1.4 target, warp modes 2/3 first render objects to the
corresponding 7.1, 7.1.2 or 7.1.4 intermediate layout, then fold the rear
surrounds into the target surround pair. The 4.0 layout also adds a rear pair
without adding centre or LFE. Other existing layouts render directly.
The intermediate layout changes speaker geometry and panning only. The trim
configuration, both balance layers and the maximum automatic position
compensation remain those of the final target. For example, 5.1.2 uses
configuration 4 and a maximum compensation of $-2.25$ dB even when its
intermediate layout is 7.1.2. With separate side and rear pairs in the
intermediate layout, floor Y is not doubled. Balance and per-object trim
are evaluated during intermediate rendering.
For intermediate outputs $L_s,R_s,L_{rs},R_{rs}$ and target surrounds
$L'_s,R'_s$:
$$
L'_s=L_s+aL_{rs}+bR_{rs},\qquad
R'_s=R_s+bL_{rs}+aR_{rs}.
$$
| Mode | $a$ | $b$ |
|---|---|---|
| 2: PLIIx | $\sqrt{3}/2$ | $1/2$ |
| 3: Standard (Lo/Ro) | $1$ | $0$ |
Both matrices add linear amplitudes without normalizing energy after the
fold. Shared front, height and LFE channels pass through to their matching
positions. The folded object gain vector enters the target-gain update and
ramp in §13 before object audio is accumulated.
The complete order is: select calibration for the target layout → apply
balance and trim while panning in the intermediate layout → fold horizontal
surrounds → update gain ramps and mix. Per-object bypass disables balance
and trim while retaining layout selection and folding. These modes are
generic speaker-rendering controls; the publicly defined E-AC-3 JOC warp
values remain 0/1. Source trajectories, ADM and binaural do not bake in the
folded result.
## 13. OAMD time alignment and gain ramps
The coded position of an OAMD update is The coded position of an OAMD update is
@@ -575,7 +795,7 @@ $$
If no new metadata update intervenes, this is equivalent to a sample-wise linear ramp of total length $KB$. If no new metadata update intervenes, this is equivalent to a sample-wise linear ramp of total length $KB$.
## 13. Final speaker mix ## 14. Final speaker mix
For target output channel $c$: For target output channel $c$:
@@ -607,9 +827,54 @@ y_{24}[n]=
\right). \right).
$$ $$
## 14. Scope of the formulas ## 15. Scope of the formulas
- The JOC matrix section covers both the dense MTX and the sparse IDX/VEC differential syntax. - The JOC matrix section covers both the dense MTX and the sparse IDX/VEC differential syntax.
- The speaker-panning section describes ordinary point objects; extent, spread, divergence, and similar modes require additional models. - The speaker-panning section describes ordinary point objects; extent, spread, divergence, and similar modes require additional models.
- Multiple OAMD position blocks must be scheduled in time order. - Multiple OAMD position blocks must be scheduled in time order.
- A limiter is separate post-processing and is not included in the mixing equations above. - A limiter is separate post-processing and is not included in the mixing equations above.
- Trim and front-back balance apply to speaker rendering only;
binaural rendering reads neither of them.
## 16. OAMD metadata application (speaker path)
When mapping object PCM to the target layout, speaker rendering applies the OAMD
object attributes in addition to the geometric panning:
- per-block integer gain `Qg` (unity 4096) and `inactive`; the gain is applied to
the target coefficients once, and the LFE uses its owning metadata object's
gain/active state;
- full basic/render status 0/1/2/3 block expansion, same-object previous-block
inheritance, and local mask updates;
- `zone_constraints_idx` region selection mapped through `[0,1,2,3,4,5,6,0]`;
`enable_elevation=0` forces z=0 before position compensation and size spread;
- `object_size_idx` 5-bit sizes, the `size_curve` piecewise mapping, and the size
kernel; the kernel is generated at runtime from region geometry with
equal-power grouped panning, the S448 weight and a power sum (20/20/8
virtual points, 20 size nodes, 35/4 position nodes; boundary rows
X(0), X(1), Y(0), Y(1), Z(1));
- `snap` first-argmax speaker snapping, `divergence` left/right point-source
combination, and element 5 high-precision position refinement;
- `default_priority`/`priority_bits` choose which size objects keep their size
when the size budget is limited;
- `isf_idx` basis matrices (signed coefficients, no per-column normalization)
with the per-basis compensation;
- `b_bed_chan_distribute` label candidates, weight normalization, missing-channel
fallback, and the direct/distribute compensation.
The ISF path orders the N ISF object audio essences into the column vector t
according to the M.U.L.Z order of `isf_config`, selects the input configuration
by `isf_idx` and the matrix M for the target output layout, and evaluates
y = M·t: each ISF channel is distributed over the speakers by the corresponding
matrix row, LFE bypasses the matrix, and each basis carries its own
compensation. The matrix coefficients and layout descriptors are in
`data/oamd/metadata_params.json`.
Gain, size, priority, position, divergence, delta and refinement values are
computed in code from formulas or fixed constants; the size kernel is generated
at runtime from region geometry with grouped equal-power panning, the S448
weight and a power sum. Bed labels, routing and reference positions are
derived from `speaker_layouts` in `src/bed_layout.py`.
ADM export and the SOFA binaural path still use object positions only and do not
apply these speaker target coefficients.
+239 -14
View File
@@ -147,13 +147,15 @@ $$
$$ $$
G_{\mathrm{clip}}= G_{\mathrm{clip}}=
1+\frac{y}{32}2^{x-4}. 2^{x-4}\left(1+\frac{y}{32}\right).
$$ $$
它并入分析端标量 $S=\frac{1}{16}G_{\mathrm{clip}}$,乘在核心 bed 的每个采样上(QMF 分析之前); 值域为 $[0.0625,15.75]$;$x=4$ 时化为 $1+y/32$。它并入分析端标量
$S=\frac{1}{16}G_{\mathrm{clip}}$,乘在核心 bed 的每个采样上(QMF 分析之前);
对象 inverse QMF 之后的 $\times16$ 抵消 $1/16$,净效果是乘 $G_{\mathrm{clip}}$,因此钳位 $[-1,1]$ 对象 inverse QMF 之后的 $\times16$ 抵消 $1/16$,净效果是乘 $G_{\mathrm{clip}}$,因此钳位 $[-1,1]$
处在增益**之后**。标量按帧保留两个槽,本帧前 256 个采样从上一帧值线性过渡到本帧值、之后为常数, 处在增益**之后**。标量按帧保留两个槽,本帧前 256 个采样从上一帧值线性过渡到本帧值、之后为常数,
复位后的第一帧直接取新值。LFE 走旁通通路,不参与该标量。 复位后的第一帧直接取新值。LFE 走独立通路,但使用同一个完整 $G_{\mathrm{clip}}$ 的逐点斜坡:
斜坡在进入 1217-sample 延迟之前乘到 LFE 采样上,见 §8。
## 3. 参数带展开与时间插值 ## 3. 参数带展开与时间插值
@@ -188,7 +190,12 @@ $$
### 3.2 单数据点插值 ### 3.2 单数据点插值
令上一帧末值为 $P_{o,c,b}$,当前目标值为 $D_{o,c,p(b)}$。对时槽 $t=0\ldots23$: 令上一帧末值为 $P_{o,c,b}$,当前目标值为 $D_{o,c,p(b)}$。对象没有 previous 状态时
(整条流第一帧或对象首次出现):单数据点情形 24 个时槽恒为 $D_{o,c,p(b)}$(硬切,
不从零淡入);双数据点情形前 12 个时槽为 $D_0$,后 12 个时槽按 §3.3 的第二段公式
从 $D_0$ 插值到 $D_1$(把该节的 $P$ 取为 $D_0$)。
有 previous 状态时,对时槽 $t=0\ldots23$:
$$ $$
\alpha_t=\frac{t+1}{24}, \alpha_t=\frac{t+1}{24},
@@ -206,11 +213,28 @@ $$
M_{o,c,b,23}=D_{o,c,p(b)}. M_{o,c,b,23}=D_{o,c,p(b)}.
$$ $$
该值随后成为下一帧的 previous 状态。 该值随后成为下一帧的 previous 状态。对象 present 由 1 变 0 后,从该帧起继续按全零矩阵发布,
并把 previous 更新为零矩阵,直到它重新 present=1。
### 3.3 多数据点 ### 3.3 多数据点
当一帧含两个数据点时,`offset_ts` 给出分段边界。每一段在上一目标和下一目标之间使用相同的线性关系;阶跃模式则在指定时槽直接切换目标。 `offset_ts` 给出各数据点开始生效的时隙序号,取原始 5 bit 值(合法范围 $0\ldots23$);
同一对象的数据点必须严格递增。默认模式(`slope_idx=0`)下,两个数据点 $D_0,D_1$ 的插值为
$$
M_{o,c,b,t}=
\begin{cases}
(1-\alpha_t)P_{o,c,b}+\alpha_tD_{0}, & t=0\ldots10,\ \alpha_t=\dfrac{t+1}{12},\\[1mm]
D_{0}, & t=11,\\[1mm]
(1-\beta_t)D_{0}+\beta_tD_{1}, & t=12\ldots23,\ \beta_t=\dfrac{t-11}{12}.
\end{cases}
$$
没有 previous 状态时同样使用上面的公式,只是把 $P_{o,c,b}$ 取为 $D_0$。
阶跃模式(`slope_idx=1`)在指定时槽直接切换目标:$t<o_0$ 保持上一帧末值 $P_{o,c,b}$
(没有 previous 时取 $D_0$),$o_0\le t<o_1$ 取 $D_0$,$t\ge o_1$ 取 $D_1$;
单数据点时 $t<o_0$ 保持 $P$、$t\ge o_0$ 取 $D_0$。
## 4. 核心 PCM 的 analysis QMF ## 4. 核心 PCM 的 analysis QMF
@@ -334,15 +358,20 @@ $$
## 8. LFE 路径 ## 8. LFE 路径
LFE 不经过对象矩阵或 inverse QMF,而是使用 1217-sample 延迟。输入与输出端的比例因子抵消后: LFE 不经过对象矩阵或 inverse QMF。核心 LFE 先乘逐点增益斜坡 $G[n]$(与 §2.4 同一个完整
$G_{\mathrm{clip}}$:$n<256$ 时 $G[n]=G_{\mathrm{old}}+n(G_{\mathrm{new}}-G_{\mathrm{old}})/256$,
$n\ge256$ 时 $G[n]=G_{\mathrm{new}}$,首帧或复位后 $G_{\mathrm{old}}=G_{\mathrm{new}}$),
再进入 1217-sample 延迟;读出后只做钳位,不再乘任何增益:
$$ $$
y_{\mathrm{LFE}}[n]= y_{\mathrm{LFE}}[n]=
\mathrm{clip}\left( \mathrm{clip}\left(
x_{\mathrm{LFE,core}}[n-1217],-1,1 x_{\mathrm{LFE,core}}[n-1217]\cdot G[n-1217],-1,1
\right). \right).
$$ $$
其中 $G[k]$ 取输入采样 $k$ 所在帧内的位置,即 $k\bmod1536$。
## 9. OAMD 坐标 ## 9. OAMD 坐标
横向和纵向网格使用 $N=62$,高度网格使用 $N=15$。量化函数为 横向和纵向网格使用 $N=62$,高度网格使用 $N=15$。量化函数为
@@ -355,7 +384,7 @@ q_N(k)=
\right). \right).
$$ $$
OAR 坐标为 归一化房间坐标为
$$ $$
u=\frac{q_1}{32768}, u=\frac{q_1}{32768},
@@ -443,14 +472,16 @@ $$
$$ $$
对于只有一对水平环绕、没有独立 side/rear 两对的布局(5.1 系列,以及 4.0 这类 对于只有一对水平环绕、没有独立 side/rear 两对的布局(5.1 系列,以及 4.0 这类
只有一对后置声道的布局),纵向坐标使用 只有一对后置声道的布局),仅在 OAMD `warp_mode=1` 时,纵向坐标使用
$$ $$
v_{\mathrm{floor}}= v_{\mathrm{floor}}=
\mathrm{clamp}(2v,0,1). \mathrm{clamp}(2v,0,1).
$$ $$
其他布局使用 $v_{\mathrm{floor}}=v$。 `warp_mode=0` 或其他布局使用 $v_{\mathrm{floor}}=v$。高度层不做该缩放。
未携带 warp 元数据时采用 0;命令行不提供覆盖选项。warp 2/3 使用 §12.5 的中间布局与折叠矩阵。
有 front/back balance 时,先求这里的分支坐标,再应用 §12.3 的 balance。
### 10.3 高度层 ### 10.3 高度层
@@ -471,6 +502,9 @@ $$
## 11. 布局位置补偿 ## 11. 布局位置补偿
本节是 trim 为 automatic 时的逐对象增益。disabled 取 1;explicit 改用 §12.2,
不能把 automatic 增益与 explicit trim 再乘一遍。
令 $N_h$ 为相关高度扬声器数,$N_f$ 为相关附加水平扬声器数: 令 $N_h$ 为相关高度扬声器数,$N_f$ 为相关附加水平扬声器数:
$$ $$
@@ -516,7 +550,165 @@ G_{\mathrm{pos}}
\mathbf G_{\mathrm{point}}. \mathbf G_{\mathrm{point}}.
$$ $$
## 12. OAMD 时间对齐与增益斜坡 ## 12. OAMD trim 与前后平衡
### 12.1 配置选择
OAMD trim element 把节目的校准数据分成九个配置,由目标布局的附加水平扬声器数
$N_f$ 与高度扬声器数 $N_h$ 选定:
$$
\kappa_f(N)=
\begin{cases}
0, & N=0,\\
1, & N\in\{2,3\},\\
2, & \text{其他},
\end{cases}
\qquad
\kappa_h(N)=
\begin{cases}
0, & N=0,\\
1, & N=2,\\
2, & \text{其他}.
\end{cases}
$$
$$
c_{\mathrm{trim}}=\kappa_f(N_f)+3\,\kappa_h(N_h).
$$
九个配置依次为“no / some / many 附加水平 × no / some / many 高度”。
trim 与前后平衡只在七种布局下施加:2.0、3.1、4.0、5.1、5.1.2、5.1.4、7.1。
更宽的布局(7.1.2、7.1.4、9.1.4、9.1.6)取 $g_t=1$、$b=0$。
warp、trim 和 balance 均从 OAMD 元数据读取。各参数的取值与作用为:
| 参数 | 取值或单位 | 行为 |
|---|---|---|
| `warp_mode` | 0 / 1 / 2 / 3 | 直接渲染 / 地面 Y 缩放 / PLIIx 折叠 / Standard(Lo/Ro)折叠 |
| `global_trim_mode` | 0 / 1 / 2 | automatic / disabled / 按目标布局配置选择;3 为保留值,明确报错 |
| $T_s$、$T_h$ | dB | 显式 surround、height trim 参数,正值按 0 处理 |
| $T_c$ | dB | 显式 centre trim 参数,保留符号 |
| $b_{\mathrm{lis}}$、$b_{\mathrm{tb}}$ | $[-1,1]$,步长 $1/16$ | 听音平面、顶部的前后 balance |
| 逐对象 trim bypass | 布尔值 | 关闭该对象的 balance 和 trim 增益,保留 warp |
全局模式为 2 时,所选配置的 default 或 disabled 标志分别采用 automatic 或 disabled;
缺失配置采用 automatic,其他配置使用其显式参数。automatic 使用 §11 的位置补偿,
disabled 使用单位增益,explicit 使用 §12.2,三种模式互斥。
初始没有 trim element 时采用 `warp_mode=0`、automatic trim。后续 payload 不含该
element 时保持上一组控制;出现新 element 时,在对应元数据事件边界更新。命令行不提供
warp 覆盖选项。
### 12.2 逐对象电平
centre、surround、height 是位置相关 trim 函数的参数,不是混音后各输出声道的固定增益。
令 $y_t$ 为原始 Y 经 balance 后的坐标,$z=\max(w,0)$:
$$
a_s=\mathrm{clamp}(2y_t,0,1)(1-z),\qquad a_h=z,
$$
$$
a_c=1-\mathrm{clamp}\bigl(10\max(|z|-0.05,\ y_t-0.05,\ |u-0.5|-0.05),0,1\bigr).
$$
explicit 状态下,surround/height 参数上限为 0 dB,centre 保留有符号电平:
$$
D=a_s\min(T_s,0)+a_h\min(T_h,0)+a_cT_c,\qquad g_t=10^{D/20}.
$$
同一个 $g_t$ 乘该对象的整条声像增益向量,然后才累加不同对象。automatic 使用 §11;
disabled 或逐对象 trim bypass 取 $g_t=1$。LFE 仍直接通过。
增益求值、声像插值和对象累加均使用 float64。
### 12.3 前后平衡
每个配置还携带两个前后平衡量:$b_{\mathrm{lis}}$ 作用于听音平面,
$b_{\mathrm{tb}}$ 作用于顶部。二者按 $1/16$ 网格量化,只在显式配置下非零。
记 $n=\mathrm{round}(16\,|b|)$、$s=n/18$,单层映射为
$$
B_b(y)=
\begin{cases}
y, & y\le 0.05\ \text{或}\ y\ge 0.95,\\[2pt]
(1+s)y-0.05s, & 0.05<y<0.5,\ b>0,\\[2pt]
(1-s)(y-0.5)+0.5+0.45s, & 0.5\le y<0.95,\ b>0,\\[2pt]
(1-s)y+0.05s, & 0.05<y<0.5,\ b<0,\\[2pt]
(1+s)(y-0.5)+0.5-0.45s, & 0.5\le y<0.95,\ b<0.
\end{cases}
$$
它把 $[0.05,0.95]$ 单调映到自身:两端不动,$y=0.5$ 处两段取值相同,斜率分别为
$1+s$ 与 $1-s$。首尾各 $5\%$ 的墙区以及 $b=0$ 的情形都原样通过,因此映射不会
越界,也无需额外钳位。
高度 $w$ 把两层混合起来:
$$
y'=(1-w)\,B_{b_{\mathrm{lis}}}(y)+w\,B_{b_{\mathrm{tb}}}(y).
$$
前后轴 $y$ 的取向见 §9,$0$ 为前、$1$ 为后;$x$ 与 $z$ 不变。
### 12.4 施加位置
记 §12.3 的高度混合 balance 映射为 $B_z(y)$,§10.2 的 warp 映射为 $W(y)$。
warp 0/1 直接渲染点对象时,分支顺序为
$$
y_f=B_z(W(y)),\qquad y_h=B_z(y),\qquad y_t=B_z(y).
$$
地面和高度声像分别使用 $y_f,y_h$,trim 函数使用 $y_t$。最后组合
$$
g_{o,c}=G_{\mathrm{object}}\,g_t(u,y_t,z)\,P_c(u,y_f,y_h,z).
$$
因此可概括为 warp → balance → trim 与声像增益合成,但不能让所有阶段共用一个
已经 warp 的 Y。balance 仅在 explicit 且未逐对象 bypass 时启用;warp 不受 trim
状态或 trim bypass 控制。源坐标保持原值,ADM 与 binaural 不预处理此变换。
地面坐标依次经过 warp、balance,再用于地面声像;顶部坐标和 trim 求值坐标只经过
balance。trim 标量乘到该对象的整条声像增益向量后,再累加到输出声道。LFE 直接通过。
### 12.5 warp 2/3 的环绕折叠
目标为 5.1、5.1.2 或 5.1.4 时,warp 2/3 先把对象渲染到对应的 7.1、7.1.2 或
7.1.4 中间布局,再将后环绕折叠到目标的环绕对。4.0 同样先增加一对后环绕,
中间布局不增加 centre 或 LFE。其他现有布局直接渲染。
中间布局只改变扬声器几何与声像分配。Trim 配置、两层 Balance 和 Auto Trim 的
最大位置补偿仍按最终目标布局选择。例如,5.1.2 使用配置 4,最大位置补偿为
$-2.25$ dB,即使其中间布局是 7.1.2,也继续使用这些参数。中间布局已有独立
side/rear 两对,地面 Y 不再乘 2;Balance 和逐对象 Trim 在中间渲染时求值。
设中间布局输出为 $L_s,R_s,L_{rs},R_{rs}$,最终目标的环绕输出为 $L'_s,R'_s$:
$$
L'_s=L_s+aL_{rs}+bR_{rs},\qquad
R'_s=R_s+bL_{rs}+aR_{rs}.
$$
| 模式 | $a$ | $b$ |
|---|---|---|
| 2:PLIIx | $\sqrt{3}/2$ | $1/2$ |
| 3:Standard(Lo/Ro) | $1$ | $0$ |
两种矩阵都按线性幅度相加,不做折叠后的能量归一化。共同的前方、高度和 LFE
声道按对应位置直接通过。折叠后的对象增益向量进入 §13 的目标增益更新与斜坡,
然后累加对象音频。
完整顺序为:按目标布局选择校准参数 → 在中间布局应用 Balance、Trim 并计算声像
增益 → 折叠水平环绕 → 更新增益斜坡与混音。逐对象 bypass 关闭 Balance 和 Trim,
但仍保留 warp 2/3 的布局选择与折叠。上述模式属于通用扬声器渲染控制;E-AC-3 JOC
公开定义的 warp 值仍为 0/1,源对象轨迹、ADM 和 binaural 不预处理折叠结果。
## 13. OAMD 时间对齐与增益斜坡
OAMD 更新的编码位置为 OAMD 更新的编码位置为
@@ -572,7 +764,7 @@ $$
如果中途没有新的 metadata 更新,该过程等价于总长度 $KB$ 的逐样本线性斜坡。 如果中途没有新的 metadata 更新,该过程等价于总长度 $KB$ 的逐样本线性斜坡。
## 13. 最终扬声器混音 ## 14. 最终扬声器混音
对目标输出声道 $c$: 对目标输出声道 $c$:
@@ -604,9 +796,42 @@ y_{24}[n]=
\right). \right).
$$ $$
## 14. 公式适用范围 ## 15. 公式适用范围
- JOC 矩阵部分同时描述 dense MTX 与 sparse IDX/VEC 两条差分语法。 - JOC 矩阵部分同时描述 dense MTX 与 sparse IDX/VEC 两条差分语法。
- 扬声器声像部分描述普通点对象;extent、spread、divergence 等模式需要额外模型。 - 扬声器声像部分描述普通点对象;extent、spread、divergence 等模式需要额外模型。
- 多个 OAMD position block 必须按其时间顺序调度。 - 多个 OAMD position block 必须按其时间顺序调度。
- limiter 属于独立后处理,不包含在上述混音公式中。 - limiter 属于独立后处理,不包含在上述混音公式中。
- trim 与前后平衡只作用于扬声器渲染;双耳渲染不读取它们。
## 16. OAMD 元数据应用(扬声器路径)
扬声器渲染把对象 PCM 映射到目标布局时,除几何声像外还应用 OAMD 的对象属性:
- 每块整数增益 `Qg`(单位增益 4096)与 `inactive`;增益在生成目标系数时乘一次,
LFE 使用所属元数据对象的增益/活动状态;
- basic/render 状态 0/1/2/3 的完整块展开、同对象前一块继承与局部 mask 更新;
- `zone_constraints_idx` 选择区域,映射为 `[0,1,2,3,4,5,6,0]`;`enable_elevation=0`
在位置补偿和尺寸扩散前令 z=0;
- `object_size_idx` 的 5 bit 尺寸、`size_curve` 分段线性映射与尺寸扩散核;
尺寸核按分组等功率声像、S448 权重和幂和,在运行时用区域几何生成:
20/20/8 个虚拟点、20 个尺寸节点、35/4 个位置节点,边界表为
X(0)、X(1)、Y(0)、Y(1)、Z(1);
- `snap` 的首个最大系数扬声器吸附、`divergence` 的左右两点合成、
element 5 的高精度位置补充;
- `default_priority`/`priority_bits` 在 size 数量预算不足时决定保留尺寸的对象;
- `isf_idx` 的 ISF 基矩阵(可含负系数,不做逐列归一化)与逐 basis 补偿;
- `b_bed_chan_distribute` 的 bed 标签候选筛选、权重归一化、缺失声道 fallback 和
直达/distribute 的补偿。
ISF 路径把 N 路 ISF object audio essences 按 `isf_config` 的 M.U.L.Z 顺序组成
列向量 t,按 `isf_idx` 选输入配置、按目标输出布局选矩阵 M,再计算 y = M·t:
每个 ISF 声道按矩阵对应的一行分配到目标扬声器,LFE 不参与矩阵、直接旁路,
每个 basis 另有补偿。矩阵系数与布局描述见 `data/oamd/metadata_params.json`。
gain、size、priority、位置、divergence 与差分/细化表由 `src/oamd_bits.py` 按
公式或固定常量计算,size 扩散核由 `src/oamd_math.py` 从区域几何生成(分组
等功率声像 + S448 权重 + 幂和)。bed 标签、路由和参考位置由
`src/bed_layout.py` 从 `speaker_layouts` 推导。
ADM 导出与 SOFA 双耳路径仍只使用对象位置,不应用这些扬声器目标系数。
+12 -35
View File
@@ -3,8 +3,8 @@
[中文](rosella_filterbank.md) · [Back to README](../README.en.md) [中文](rosella_filterbank.md) · [Back to README](../README.en.md)
The Rosella `.personalized_headphone` path can run either of two 64-QMF / The Rosella `.personalized_headphone` path can run either of two 64-QMF /
77-hybrid filterbanks: the shared public bank (the default) or the Rosella bank 77-hybrid filterbanks: the 13-tap public bank (the default) or the 11-tap Rosella
captured from the reference renderer. The SOFA and `.jochrtf` paths always use bank. The SOFA and `.jochrtf` paths always use
the public bank and never consult the selector. the public bank and never consult the selector.
## Which bank runs ## Which bank runs
@@ -26,30 +26,21 @@ and never share it. Any other value is rejected, and the option is an error when
the binaural input is not a Rosella model: a SOFA or `.jochrtf` render must not the binaural input is not a Rosella model: a SOFA or `.jochrtf` render must not
silently ignore it. silently ignore it.
## Why the banks differ ## Bank parameters
The reference headphone renderer selects its analysis and synthesis callbacks | Parameter | 11-tap Rosella bank | 13-tap public bank |
during preparation, after initialization. Capturing only the initial callbacks
identified the wrong filterbank variant. Runtime buffer captures gave these
parameters:
| Parameter | Captured Rosella bank | Public bank |
| --- | --- | --- | | --- | --- | --- |
| QMF prototype | 640 asymmetric coefficients | polyphase reorder of the public 640-tap prototype | | QMF prototype | 640 asymmetric coefficients | polyphase reorder of the public 640-tap prototype |
| Low-band hybrid filter | 11 taps | 13 taps | | Low-band hybrid filter | 11 taps | 13 taps |
| High-band hybrid delay | 3 slots | 6 slots | | High-band hybrid delay | 3 slots | 6 slots |
| Analysis/synthesis latency | 448 samples | 961 samples | | Analysis/synthesis latency | 448 samples | 961 samples |
The wrong variant produced a frequency-dependent room-tail error. With the The selector does not change room coefficients, direct HRTF calculation, phase
captured room send as input, the existing room recursion already matched the mapping or output EQ. Bank `1` is the default.
reference, so the fix changes only the filterbank: not the room coefficients,
the direct HRTF calculation, the phase mapping, or the output EQ. The public
bank stayed as bank `1` so historical renders remain reproducible; the captured
bank is bank `2`.
## Implementation ## Implementation
`rosella_captured_filterbank.py` reproduces the four half-steps of the captured `rosella_captured_filterbank.py` executes the four half-steps of the 11-tap
bank in float64/complex128: bank in float64/complex128:
- QMF analysis keeps a 9-slot ring history, reads `(position + tap + 1) mod 10`, - QMF analysis keeps a 9-slot ring history, reads `(position + tap + 1) mod 10`,
@@ -60,30 +51,16 @@ bank in float64/complex128:
- QMF synthesis keeps a 9-slot history and accumulates 10 prototype-matched - QMF synthesis keeps a 9-slot history and accumulates 10 prototype-matched
lags, taking the imaginary part on odd lags. lags, taking the imaginary part on odd lags.
`data/rosella_captured_kernels.npz` stores the captured prototype `data/rosella_captured_kernels.npz` stores the 11-tap bank's prototype
(`float32[64,10]`), rotation (`complex64[64]`) and hybrid coefficients (`float32[64,10]`), rotation (`complex64[64]`) and hybrid coefficients
(`complex64[22]`); they are promoted to float64/complex128 on load. The values (`complex64[22]`); they are promoted to float64/complex128 on load and contain
are the ones the C++ core compiles in as `src/hrtf/rosella_filterbank_tables.h` no personalized HRTF or room tuning.
and contain no personalized HRTF or room tuning.
The startup skip and the flush capacity both come from the selected bank (961 or The startup skip and the flush capacity both come from the selected bank (961 or
448 samples). `.report.json` records the actual choice in 448 samples). `.report.json` records the actual choice in
`binaural_renderer_backend`, as `filterbank`, `filterbank_name` and `binaural_renderer_backend`, as `filterbank`, `filterbank_name` and
`latency_compensated_samples`. `latency_compensated_samples`.
## Validation The public filterbank formulas and `.jochrtf` direction field are described in
`docs/binaural.md`; the 11-tap bank only affects the Rosella
The local regression script lives in `tests/`, which this repository does not
track (like the rest of its local checks). It covers:
- reference vectors for the four captured half-steps, the same samples the C++
core checks, generated by an independent NumPy implementation, to within
`1e-10` per sample;
- an impulse roundtrip peak at 961 and 448 samples for the two banks;
- chunk invariance (including zero-length and unequal chunks) and `reset()`;
- independence: processing one bank never changes the other bank's state;
- rejection of out-of-range values.
The public filterbank formulas and `.jochrtf` direction field described in
`docs/binaural.md` are unchanged; the captured bank only affects the Rosella
path. path.
+9 -26
View File
@@ -3,7 +3,7 @@
[English](rosella_filterbank.en.md) · [返回 README](../README.md) [English](rosella_filterbank.en.md) · [返回 README](../README.md)
Rosella `.personalized_headphone` 路径可以在两套 64-QMF / 77-hybrid 滤波器组之间选择: Rosella `.personalized_headphone` 路径可以在两套 64-QMF / 77-hybrid 滤波器组之间选择:
公开滤波器组(默认)与从参考渲染器捕获的 Rosella 滤波器组。SOFA 与 `.jochrtf` 路径 13-tap 公开滤波器组(默认)与 11-tap Rosella 滤波器组。SOFA 与 `.jochrtf` 路径
始终使用公开滤波器组,不经过该选择。 始终使用公开滤波器组,不经过该选择。
## 选择哪一套 ## 选择哪一套
@@ -21,25 +21,20 @@ QMF 合成),因此切换会同时改变系数、高频延迟、历史缓冲
flush 容量;分析与合成两级各自持有状态,互不共享。其它取值直接报错;非 Rosella flush 容量;分析与合成两级各自持有状态,互不共享。其它取值直接报错;非 Rosella
输入(SOFA、`.jochrtf`)给出该选项也报错,不会静默忽略。 输入(SOFA、`.jochrtf`)给出该选项也报错,不会静默忽略。
## 为什么两套不同 ## 两套参数
参考头戴渲染器在初始化之后的准备阶段才选定 analysis/synthesis 回调,只捕获初始化 | 参数 | 11-tap Rosella | 13-tap 公开版 |
时的回调会拿到错误的滤波器组变体。运行时缓冲捕获得到的参数是:
| 参数 | Rosella 捕获版 | 公开版 |
| --- | --- | --- | | --- | --- | --- |
| QMF prototype | 640 个非对称系数 | 公开 640-tap prototype 的多相重排 | | QMF prototype | 640 个非对称系数 | 公开 640-tap prototype 的多相重排 |
| 低频 hybrid 滤波 | 11 taps | 13 taps | | 低频 hybrid 滤波 | 11 taps | 13 taps |
| 高频 hybrid delay | 3 slots | 6 slots | | 高频 hybrid delay | 3 slots | 6 slots |
| analysis/synthesis 延迟 | 448 samples | 961 samples | | analysis/synthesis 延迟 | 448 samples | 961 samples |
用错变体会在 room 尾音上引入随频率变化的误差;把捕获到的 room send 作为输入时, 该选项不改变 room 系数、direct HRTF、相位映射或输出 EQ。默认选择 `1`。
现有 room 递推已经能对上参考,因此修复只更换滤波器组,不改 room 系数、direct HRTF、
相位映射或输出 EQ。公开版是 `1`,捕获版是 `2`;默认保留公开版,便于与历史输出对齐。
## 实现 ## 实现
`rosella_captured_filterbank.py` 用 float64/complex128 复现捕获版的四个 half-step: `rosella_captured_filterbank.py` 用 float64/complex128 执行 11-tap 滤波器组的四个 half-step:
- QMF 分析保留 9 slot 环形历史,按 `(position + tap + 1) mod 10` 读取,输出 - QMF 分析保留 9 slot 环形历史,按 `(position + tap + 1) mod 10` 读取,输出
`spectrum[2b] = F[b]/64`、`spectrum[2b+1] = conj(F[63-b])/64`; `spectrum[2b] = F[b]/64`、`spectrum[2b+1] = conj(F[63-b])/64`;
@@ -47,25 +42,13 @@ flush 容量;分析与合成两级各自持有状态,互不共享。其它
band 1 与 band 3 取共轭;高频带原样延迟 3 slot; band 1 与 band 3 取共轭;高频带原样延迟 3 slot;
- QMF 合成保留 9 slot 历史,用 10 个 lag 的 prototype 匹配累加,奇数 lag 取虚部。 - QMF 合成保留 9 slot 历史,用 10 个 lag 的 prototype 匹配累加,奇数 lag 取虚部。
`data/rosella_captured_kernels.npz` 保存捕获版的 prototype(`float32[64,10]`)、 `data/rosella_captured_kernels.npz` 保存 11-tap 滤波器组的 prototype(`float32[64,10]`)、
rotation(`complex64[64]`)与 hybrid 系数(`complex64[22]`);载入后提升为 rotation(`complex64[64]`)与 hybrid 系数(`complex64[22]`);载入后提升为
float64/complex128。这些数值与 C++ 内核编译进去的 float64/complex128,不包含个性化 HRTF 或 room 调参。
`src/hrtf/rosella_filterbank_tables.h` 相同,不包含个性化 HRTF 或 room 调参。
运行时的起始延迟跳过与 flush 容量都取自所选滤波器组:`961` 或 `448` samples, 运行时的起始延迟跳过与 flush 容量都取自所选滤波器组:`961` 或 `448` samples,
`.report.json` 的 `binaural_renderer_backend` 用 `filterbank`、`filterbank_name` 与 `.report.json` 的 `binaural_renderer_backend` 用 `filterbank`、`filterbank_name` 与
`latency_compensated_samples` 记录实际选择。 `latency_compensated_samples` 记录实际选择。
## 验证 `docs/binaural.md` 描述的公开滤波器组公式与 `.jochrtf` 方向和公开版一致;11-tap 版本
只作用于 Rosella 路径。
本地回归脚本放在 `tests/`(该目录不随仓库跟踪,与其余本地验证脚本一致),覆盖:
- 捕获版四个 half-step 的参考向量(与 C++ 内核 `tests/rosella_filterbank_vectors.h`
同一组、由独立 NumPy 实现生成),逐样本误差小于 `1e-10`;
- 两套滤波器组的脉冲往返延迟分别落在 `961` 与 `448` samples;
- 分块处理(含 0 长度与不等长切分)与 `reset()` 不变性;
- 两套滤波器组互不干扰:交叉处理一个 bank 不改变另一个 bank 的状态;
- 越界取值被拒绝。
`docs/binaural.md` 描述的公开滤波器组公式与 `.jochrtf` 方向和公开版一致;捕获版只
作用于 Rosella 路径。
+6 -2
View File
@@ -699,8 +699,10 @@ def main(argv=None):
timings["create_binaural_renderer"] = 0.0 timings["create_binaural_renderer"] = 0.0
layout = get_speaker_layout(args.speaker_layout) layout = get_speaker_layout(args.speaker_layout)
speaker_name = speaker_layout_display_name(layout) speaker_name = speaker_layout_display_name(layout)
speaker_meta = oamd_tracks.read_atmos_meta(index)
speaker_decoder, speaker_backend_info = create_speaker_renderer( speaker_decoder, speaker_backend_info = create_speaker_renderer(
layout, bed_channels=getattr(index, "bed_channels", ())) layout, bed_channels=getattr(index, "bed_channels", ()),
trim_meta=speaker_meta)
print(f"[speaker] layout={speaker_name} backend={speaker_backend_info['name']} " print(f"[speaker] layout={speaker_name} backend={speaker_backend_info['name']} "
f"channels={layout.channel_count}", flush=True) f"channels={layout.channel_count}", flush=True)
spool = SpeakerPcmSpool( spool = SpeakerPcmSpool(
@@ -844,9 +846,11 @@ def main(argv=None):
f"samples={binaural_clip_info['kept_samples']}") f"samples={binaural_clip_info['kept_samples']}")
else: else:
timings["create_binaural_renderer"] = 0.0 timings["create_binaural_renderer"] = 0.0
atmos_meta = oamd_tracks.read_atmos_meta(index)
master = adm_assemble.StreamingMaster( master = adm_assemble.StreamingMaster(
output, duration_sec, rate=RATE, output, duration_sec, rate=RATE,
joc_binaural_mode=adm_atmos.JOC_BINAURAL_MODES[args.binaural_mode]) joc_binaural_mode=adm_atmos.JOC_BINAURAL_MODES[args.binaural_mode],
atmos_meta=atmos_meta)
try: try:
render_seconds, renderer_backend, render_breakdown = timed_call( render_seconds, renderer_backend, render_breakdown = timed_call(
timings, "render_and_stream", variant_call, timings, "render_and_stream", variant_call,
+6 -3
View File
@@ -71,13 +71,14 @@ class StreamingMaster:
""" """
def __init__(self, out_path, duration_sec, rate=48000, block_samples=131072, def __init__(self, out_path, duration_sec, rate=48000, block_samples=131072,
joc_binaural_mode=4): joc_binaural_mode=4, atmos_meta=None):
if block_samples < 1536: if block_samples < 1536:
raise ValueError("block_samples must be at least one E-AC-3 frame") raise ValueError("block_samples must be at least one E-AC-3 frame")
self.out_path = os.fspath(out_path) self.out_path = os.fspath(out_path)
self.duration_sec = float(duration_sec) self.duration_sec = float(duration_sec)
self.rate = int(rate) self.rate = int(rate)
self.joc_binaural_mode = joc_binaural_mode self.joc_binaural_mode = joc_binaural_mode
self.atmos_meta = atmos_meta
self._sink = adm_atmos.Sink25(self.out_path, 25, self.rate) self._sink = adm_atmos.Sink25(self.out_path, 25, self.rate)
self._buffer = np.empty((int(block_samples), 25), dtype=np.float32) self._buffer = np.empty((int(block_samples), 25), dtype=np.float32)
self._used = 0 self._used = 0
@@ -113,10 +114,12 @@ class StreamingMaster:
from . import adm_serializer from . import adm_serializer
except ImportError: except ImportError:
import adm_serializer import adm_serializer
axml = adm_serializer.build_axml(kf_tracks, self.duration_sec) axml = adm_serializer.build_axml(kf_tracks, self.duration_sec,
atmos_meta=self.atmos_meta)
chna = adm_atmos.build_chna() chna = adm_atmos.build_chna()
dbmd = adm_atmos.build_dbmd( dbmd = adm_atmos.build_dbmd(
25, joc_binaural_mode=self.joc_binaural_mode) 25, joc_binaural_mode=self.joc_binaural_mode,
atmos_meta=self.atmos_meta)
trajectory_blocks = sum(len(track[1]) for track in kf_tracks) trajectory_blocks = sum(len(track[1]) for track in kf_tracks)
self.metadata_info = { self.metadata_info = {
"axml_bytes": len(axml), "axml_bytes": len(axml),
+59 -3
View File
@@ -32,6 +32,10 @@ JOC_BINAURAL_MODES = {
JOC_BINAURAL_MODE_DEFAULT = "unspecified" JOC_BINAURAL_MODE_DEFAULT = "unspecified"
def q_to_adm_xyz(q1, q2, q3): def q_to_adm_xyz(q1, q2, q3):
"""OAMD 的 q1/q2/q3 编码 → ADM 笛卡尔坐标。
posX/posY 为 62 级量化,posZ 为 15 级;ADM 的 Y 轴与 OAMD 编码方向相反。
"""
posX = min(1.0, round(q1 * 62 / 32767.0) / 62.0) posX = min(1.0, round(q1 * 62 / 32767.0) / 62.0)
posY = min(1.0, round(q2 * 62 / 32767.0) / 62.0) posY = min(1.0, round(q2 * 62 / 32767.0) / 62.0)
posZ = round(q3 * 15 / 32767.0) / 15.0 posZ = round(q3 * 15 / 32767.0) / 15.0
@@ -195,8 +199,34 @@ def _checksum(seg):
s += b s += b
return (~s + 1) & 0xFF return (~s + 1) & 0xFF
def build_dbmd(object_count=25, joc_binaural_mode=4): def _balance_byte(value):
"""仅覆盖 segment 10 中 JOC object slots 10..24 的 mode 低 3 bit。""" """trim 记录里的 balance 字段:bit7 为符号,bit0-6 为幅度。
bal3d_y 的取值落在 [0, 1],7 bit 幅度即该区间上的定点表示,步长 1/127。
"""
value = float(value)
magnitude = min(int(round(abs(value) * 127.0)), 127)
return magnitude | (0x80 if value < 0 else 0)
def warp_mode(atmos_meta):
"""OAMD trim element 的 ``warp_mode`` → 下混模式码。
dbmd 的 warp 字段、ADM 的 ``DolbyWarpMode`` 和 DAMF 的 ``WarpMode`` 共用
一套编号:0 normal、1 warping、2 downmix_pl2x、3 downmix_loro,4 表示未
指定。来源只有 2 bit 的 OAMD 字段,因此取值落在 0..3。
"""
if not atmos_meta:
return 0
return max(0, min(3, int(atmos_meta.get("warp_mode", 0))))
def build_dbmd(object_count=25, joc_binaural_mode=4, atmos_meta=None):
"""segment 10:9 个 trim 配置 + 逐轨 bypass + JOC object slots 10..24 的 mode。
含显式值的配置写成显式记录(3 个 float32 + 2 个 balance 字节),其余保持
自动标志。
"""
mode = operator.index(joc_binaural_mode) mode = operator.index(joc_binaural_mode)
if mode not in JOC_BINAURAL_MODES.values(): if mode not in JOC_BINAURAL_MODES.values():
raise ValueError(f"invalid JOC binaural render mode: {mode}") raise ValueError(f"invalid JOC binaural render mode: {mode}")
@@ -215,6 +245,9 @@ def build_dbmd(object_count=25, joc_binaural_mode=4):
at[103] = 0x03 at[103] = 0x03
at[106] = 0x01 at[106] = 0x01
at[111] = 0x22; at[112] = 0xFF at[111] = 0x22; at[112] = 0xFF
# at[152] 高 2 位是 bed_distribution、低 3 位是 warp;这里只写 warp,
# 其余位保持模板原值。
at[152] = (at[152] & 0xF8) | warp_mode(atmos_meta)
out.append(9); out += struct.pack("<H", 248); out += bytes(at) out.append(9); out += struct.pack("<H", 248); out += bytes(at)
out.append(_checksum(at)) out.append(_checksum(at))
ob = bytearray(5 + 262 + object_count) ob = bytearray(5 + 262 + object_count)
@@ -230,7 +263,30 @@ def build_dbmd(object_count=25, joc_binaural_mode=4):
# default is automatic, so a freshly rendered master opens with automatic # default is automatic, so a freshly rendered master opens with automatic
# trims in the renderer. # trims in the renderer.
for layout in range(9): for layout in range(9):
ob[7 + 15 * layout] |= 1 base = 7 + 15 * layout
record = None
if atmos_meta is not None:
configs = atmos_meta.get("configs") or ()
if layout < len(configs):
record = configs[layout]
explicit = isinstance(record, dict) and not record.get("default")
if not explicit:
ob[base] |= 1 # bit0 = 1:由渲染器推导
continue
ob[base] &= 0xFE # bit0 = 0:显式
if record.get("disabled"):
trims = (0.0, 0.0, 0.0)
bal3d_y_tb = bal3d_y_lis = 0.0
else:
trims = (record.get("centre", 0.0), record.get("surround", 0.0),
record.get("height", 0.0))
bal3d_y_tb = record.get("bal3d_y_tb", 0.0)
bal3d_y_lis = record.get("bal3d_y_lis", 0.0)
for index, value in enumerate(trims):
offset = base + 1 + 4 * index
ob[offset:offset + 4] = struct.pack("<f", float(value))
ob[base + 13] = _balance_byte(bal3d_y_tb)
ob[base + 14] = _balance_byte(bal3d_y_lis)
object_modes = 4 + 2 + 1 + 9 * 15 + object_count object_modes = 4 + 2 + 1 + 9 * 15 + object_count
for i in range(10, min(object_count, 10 + N_OBJ)): for i in range(10, min(object_count, 10 + N_OBJ)):
ob[object_modes + i] = (ob[object_modes + i] & 0xF8) | mode ob[object_modes + i] = (ob[object_modes + i] & 0xF8) | mode
+100 -2
View File
@@ -23,8 +23,101 @@ def ts(seconds):
def esc(v): def esc(v):
return (str(v).replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;")) return (str(v).replace("&", "&amp;").replace("<", "&lt;").replace(">", "&gt;"))
def build_axml(obj_tracks, duration_sec): def _fmt_float(value):
"""obj_tracks: [(name, [(rtime, x, y, z, dur), ...]) ×15]""" return f"{float(value):.6f}"
def _trim_records(atmos_meta):
"""把 OAMD trim element 展开成 DolbyTrimModeSetup 的 records。
``configs`` 每项有三种形态:
* ``{"default": True}`` —— 交给渲染器推导,不写任何值子元素;
* ``{"disabled": True}`` —— 该配置不施加 trim,写全 0;
* 含 centre/surround/height/bal3d_y_tb/bal3d_y_lis 的显式配置。
缺省字段一律补 0。
"""
lines = ["<DolbyTrimModeSetup>"]
for index, record in enumerate(atmos_meta.get("configs") or ()):
if record is None:
continue
lines.append("<records>")
lines.append(f"<type>{index}</type>")
if isinstance(record, dict) and not record.get("default"):
if record.get("disabled"):
values = {"centerTrim": 0.0, "surroundTrim": 0.0,
"heightTrim": 0.0, "balanceOverhead": 0.0,
"balanceListener": 0.0}
else:
values = {
"centerTrim": record.get("centre", 0.0),
"surroundTrim": record.get("surround", 0.0),
"heightTrim": record.get("height", 0.0),
"balanceOverhead": record.get("bal3d_y_tb", 0.0),
"balanceListener": record.get("bal3d_y_lis", 0.0),
}
for name, value in values.items():
lines.append(f"<{name}>{_fmt_float(value)}</{name}>")
lines.append("</records>")
lines.append("</DolbyTrimModeSetup>")
return lines
def _warp_xml_value(atmos_meta):
"""OAMD 的 warp_mode → ADM ``DolbyWarpMode`` 取值。
warp_mode 选择下混/渲染模式,四档依次为 Direct render、Direct render
with room balance、Dolby Pro Logic IIx、Standard (Lo/Ro),ADM 沿用同一
编号,因此直接照搬码流值。
"""
return max(0, min(3, int(atmos_meta.get("warp_mode", 0))))
def _atmos_master_block(atmos_meta):
"""Dolby 扩展容器:内容信息 + trim 记录 + warp。"""
lines = ["<DolbyAtmosMasterFile>"]
warp = _warp_xml_value(atmos_meta)
lines.append("<DolbyMasterContentInfo>")
lines.append(f"<render_warp_mode>{warp}</render_warp_mode>")
lines.append("</DolbyMasterContentInfo>")
lines.extend(_trim_records(atmos_meta))
lines.append(f"<DolbyWarpMode>{warp}</DolbyWarpMode>")
lines.append("</DolbyAtmosMasterFile>")
return lines
def _object_meta_lines(meta):
"""把 OAMD 的逐对象渲染参数写成 ADM 的 audioBlockFormat 子元素。
只写 ADM 能表达的字段:size → width/height/depth(相对单位 0..1),
gain → gain(线性),divergence → objectDivergence(笛卡尔坐标系用
positionRange)。缺省值(size 0、gain 1.0、divergence 0)省略。
"""
if not meta:
return []
lines = []
size = meta.get("size") or (0.0, 0.0, 0.0)
# OAMD 的 size 三轴顺序为 width, depth, height。
width, depth, height = size
if width > 0.0 or depth > 0.0 or height > 0.0:
lines.append(f'<width>{width:.6f}</width>')
lines.append(f'<height>{height:.6f}</height>')
lines.append(f'<depth>{depth:.6f}</depth>')
gain = meta.get("gain")
if gain is not None and abs(float(gain) - 1.0) > 1e-9:
lines.append(f'<gain gainUnit="linear">{float(gain):.6f}</gain>')
divergence = meta.get("divergence")
if divergence is not None and float(divergence) > 0.0:
lines.append(f'<objectDivergence positionRange="{float(divergence):.6f}"/>')
return lines
def build_axml(obj_tracks, duration_sec, atmos_meta=None):
"""obj_tracks: [(name, [(rtime, x, y, z, dur, interpolation, meta), ...]) ×15]
``meta`` 可省略;给出时写 width/height/depth、gain、objectDivergence。
``atmos_meta`` 为 ``oamd_bits.parse_trim`` 的结果;给出时把 trim 与 warp
写进 ADM。
"""
w = [] w = []
a = w.append a = w.append
a('<?xml version="1.0" encoding="utf-8"?>') a('<?xml version="1.0" encoding="utf-8"?>')
@@ -87,6 +180,7 @@ def build_axml(obj_tracks, duration_sec):
for k, keyframe in enumerate(kfs): for k, keyframe in enumerate(kfs):
t, x, y, z, dur = keyframe[:5] t, x, y, z, dur = keyframe[:5]
interpolation = keyframe[5] if len(keyframe) > 5 else 0.0 interpolation = keyframe[5] if len(keyframe) > 5 else 0.0
meta = keyframe[6] if len(keyframe) > 6 else None
a(f'<audioBlockFormat audioBlockFormatID="AB_0003{0x1001 + i:04x}_{k + 1:08x}" ' a(f'<audioBlockFormat audioBlockFormatID="AB_0003{0x1001 + i:04x}_{k + 1:08x}" '
f'rtime="{ts(t)}" duration="{ts(dur)}">') f'rtime="{ts(t)}" duration="{ts(dur)}">')
a('<cartesian>1</cartesian>') a('<cartesian>1</cartesian>')
@@ -94,6 +188,8 @@ def build_axml(obj_tracks, duration_sec):
a(f'<position coordinate="Y">{y:.10f}</position>') a(f'<position coordinate="Y">{y:.10f}</position>')
if z != 0: if z != 0:
a(f'<position coordinate="Z">{z:.10f}</position>') a(f'<position coordinate="Z">{z:.10f}</position>')
for line in _object_meta_lines(meta):
a(line)
a(f'<jumpPosition interpolationLength="{interpolation:.5f}">1</jumpPosition>') a(f'<jumpPosition interpolationLength="{interpolation:.5f}">1</jumpPosition>')
a('</audioBlockFormat>') a('</audioBlockFormat>')
a('</audioChannelFormat>') a('</audioChannelFormat>')
@@ -131,6 +227,8 @@ def build_axml(obj_tracks, duration_sec):
a(f'<audioPackFormatIDRef>AP_0003{0x1001 + i:04x}</audioPackFormatIDRef>') a(f'<audioPackFormatIDRef>AP_0003{0x1001 + i:04x}</audioPackFormatIDRef>')
a(f'<audioTrackFormatIDRef>AT_0003{0x1001 + i:04x}_01</audioTrackFormatIDRef>') a(f'<audioTrackFormatIDRef>AT_0003{0x1001 + i:04x}_01</audioTrackFormatIDRef>')
a('</audioStreamFormat>') a('</audioStreamFormat>')
if atmos_meta:
w.extend(_atmos_master_block(atmos_meta))
a('</audioFormatExtended></format></coreMetadata>') a('</audioFormatExtended></format></coreMetadata>')
a('</ebuCoreMain>') a('</ebuCoreMain>')
return ''.join(w).encode('utf-8') return ''.join(w).encode('utf-8')
+2 -2
View File
@@ -173,8 +173,8 @@ def main():
import argparse import argparse
ap = argparse.ArgumentParser() ap = argparse.ArgumentParser()
ap.add_argument("files", nargs="+") ap.add_argument("files", nargs="+")
ap.add_argument("--axml-file", default=None, help="用该文件内容替换 wav 内 axml(对照实验)") ap.add_argument("--axml-file", default=None, help="用该文件内容替换 wav 内 axml")
ap.add_argument("--chna-file", default=None, help="用该文件内容替换 wav 内 chna(对照实验)") ap.add_argument("--chna-file", default=None, help="用该文件内容替换 wav 内 chna")
a = ap.parse_args() a = ap.parse_args()
ax_o = open(a.axml_file, "rb").read() if a.axml_file else None ax_o = open(a.axml_file, "rb").read() if a.axml_file else None
ch_o = open(a.chna_file, "rb").read() if a.chna_file else None ch_o = open(a.chna_file, "rb").read() if a.chna_file else None
+152 -12
View File
@@ -4,6 +4,9 @@
那一对落到 ADM 骨架的槽位。JOC 的 DSP 不读取这些信息(规格 §10.1),它只用 那一对落到 ADM 骨架的槽位。JOC 的 DSP 不读取这些信息(规格 §10.1),它只用
``(N, sigma)``。 ``(N, sigma)``。
""" """
import math
from functools import lru_cache
from emdf import expand_chanmap from emdf import expand_chanmap
# JOC 核心通道序列的前 5 位(规格 §2):L R C Ls Rs。 # JOC 核心通道序列的前 5 位(规格 §2):L R C Ls Rs。
@@ -79,16 +82,17 @@ BED_CHANNEL_NAMES = {
"RC_TBL": "Ltr", "RC_TBR": "Rtr", "RC_LW": "Lw", "RC_RW": "Rw", "RC_TBL": "Ltr", "RC_TBR": "Rtr", "RC_LW": "Lw", "RC_RW": "Rw",
} }
# 每个 bed 声道的固定方位(ADM 笛卡尔坐标,x 向右、y 向前、z 向上),取自 # 每个 bed 声道的固定方位(ADM 笛卡尔坐标,x 向右、y 向前、z 向上):speakerLabel
# ETSI TS 103 420 V1.2.1 clause B.2.1.4 Table B.10:speakerLabel 各自对应唯一的 # 各自对应唯一的 position(X,Y,Z),与目标扬声器布局无关。bed 对象在 OAMD 里没有位置
# position(X,Y,Z),与目标扬声器布局无关。bed 对象在 OAMD 里没有位置字段,位置只由 # 字段,位置只由声明的声道标签决定。
# 声明的声道标签决定。 # 天空声道的 X 取墙边(±1):点声源的横向声像只在扬声器跨度的端点之间分摊,取端点仍
# Table B.10 把 Lw/Rw 的 Y 记作 0,677419,即 ADM 62 级网格上的 42/62 = 21/31。 # 落在输出布局顶部声道的满功率区里,与顶部声道互相下混的结果一致。Lw/Rw 的 Y 记作
# 0,677419,即 42/62 = 21/31。LFE/LFE2 不参与摆位,位置记为原点。
BED_CHANNEL_ADM_POSITION = { BED_CHANNEL_ADM_POSITION = {
"RC_L": (-1.0, 1.0, 0.0), "RC_L": (-1.0, 1.0, 0.0),
"RC_R": (1.0, 1.0, 0.0), "RC_R": (1.0, 1.0, 0.0),
"RC_C": (0.0, 1.0, 0.0), "RC_C": (0.0, 1.0, 0.0),
"RC_LFE": (-1.0, 1.0, -1.0), "RC_LFE": (0.0, 0.0, 0.0),
"RC_LS": (-1.0, 0.0, 0.0), "RC_LS": (-1.0, 0.0, 0.0),
"RC_RS": (1.0, 0.0, 0.0), "RC_RS": (1.0, 0.0, 0.0),
"RC_LB": (-1.0, -1.0, 0.0), "RC_LB": (-1.0, -1.0, 0.0),
@@ -101,12 +105,143 @@ BED_CHANNEL_ADM_POSITION = {
"RC_TBR": (1.0, -1.0, 1.0), "RC_TBR": (1.0, -1.0, 1.0),
"RC_LW": (-1.0, 21.0 / 31.0, 0.0), "RC_LW": (-1.0, 21.0 / 31.0, 0.0),
"RC_RW": (1.0, 21.0 / 31.0, 0.0), "RC_RW": (1.0, 21.0 / 31.0, 0.0),
"RC_LFE2": (1.0, 1.0, -1.0), "RC_LFE2": (0.0, 0.0, 0.0),
} }
# LFE 不是空间声源:它的 PCM 走独立通路,不参与对象摆位。 # LFE 不是空间声源:它的 PCM 走独立通路,不参与对象摆位。
BED_LFE_LABELS = frozenset(("RC_LFE", "RC_LFE2")) BED_LFE_LABELS = frozenset(("RC_LFE", "RC_LFE2"))
# 参考位置网格:bed 标签/输出 slots 的最近邻匹配使用这一组固定的 Q15 点。
BED_REFERENCE_POSITIONS_Q15 = (
(0, 0, 0), (32767, 0, 0), (16384, 0, 0), (-1, -1, -1),
(0, 16384, 0), (32767, 16384, 0), (0, 32767, 0), (32767, 32767, 0),
(16384, 32767, 0), (5285, 0, 0), (27483, 0, 0), (0, 5285, 0),
(32767, 5285, 0), (7928, 0, 32767), (24840, 0, 32767),
(7928, 7928, 32767), (24840, 7928, 32767), (7928, 16384, 32767),
(24840, 16384, 32767), (7928, 24840, 32767), (24840, 24840, 32767),
(7928, 32767, 32767), (24840, 32767, 32767), (11099, 0, 0),
(21669, 0, 0), (0, 11099, 0), (32767, 11099, 0), (0, 21669, 0),
(32767, 21669, 0), (0, 27483, 0), (32767, 27483, 0), (5285, 32767, 0),
(27483, 32767, 0), (11099, 32767, 0), (21669, 32767, 0),
(16384, 0, 32767), (16384, 16384, 32767), (16384, 32767, 32767),
)
# 自动 trim 的每个布局基准衰减,单位 dB。
BED_LAYOUT_ATTENUATION_DB = {
"20": -4.5, "31": -4.5, "40": -3.0, "51": -3.0, "60": -1.5,
"71": -1.5, "512": -2.25, "514": -1.5, "712": -0.75,
"714": 0.0, "914": 0.0, "916": 0.0,
}
def _channel_positions(layout):
"""Channel index -> Q15 room position (LFE gets the special sentinel)."""
found = {}
for region in layout.regions:
for speaker, coordinate in zip(region.speaker_ids,
region.coordinates_q15):
found.setdefault(int(speaker), tuple(coordinate))
result = []
for index, channel in enumerate(layout.channels):
if channel == "LFE":
result.append((-1, -1, -1))
else:
result.append(found.get(index, (-1, -1, -1)))
return result
def _nearest_reference(coordinate):
"""Index of the closest entry in BED_REFERENCE_POSITIONS_Q15."""
if tuple(coordinate) == (-1, -1, -1):
return 3
position = (coordinate[0] / 32768.0, coordinate[1] / 32768.0,
coordinate[2] / 32768.0)
best = 3
best_distance = float("inf")
for index, candidate in enumerate(BED_REFERENCE_POSITIONS_Q15):
if candidate == (-1, -1, -1):
continue
distance = math.dist(position,
(candidate[0] / 32768.0,
candidate[1] / 32768.0,
candidate[2] / 32768.0))
if distance < best_distance:
best = index
best_distance = distance
return best
def _nearest_channel(desired, positions, tolerance=0.05):
"""Channel index nearest to desired (u, v, w), or None if too far."""
best = None
best_distance = float("inf")
for index, coordinate in enumerate(positions):
if coordinate == (-1, -1, -1):
continue
candidate = (coordinate[0] / 32768.0, coordinate[1] / 32768.0,
coordinate[2] / 32768.0)
distance = math.dist(desired, candidate)
if distance < best_distance:
best = index
best_distance = distance
return best if best_distance <= tolerance else None
@lru_cache(maxsize=32)
def bed_output_description(layout):
"""Derive the bed routing description from a speaker layout.
The description is keyed by the standard bed label order and contains the
output reference-position list, per-label slot routing and the automatic
trim attenuation for the layout.
"""
channels = tuple(layout.channels)
positions = _channel_positions(layout)
output_labels = [_nearest_reference(position) for position in positions]
label_to_slot = []
direct_output_labels = []
for label in (item for pair in BED_LAYOUT_ORDER
for item in pair.split("/")):
name = BED_CHANNEL_NAMES.get(label)
slot = None
if name in channels:
slot = channels.index(name)
elif label == "RC_LFE2" and "LFE" in channels:
slot = channels.index("LFE")
else:
desired = adm_xyz_to_render_coordinates(
*BED_CHANNEL_ADM_POSITION[label])
slot = _nearest_channel(desired, positions)
if slot is None:
label_to_slot.append(-1 if label in BED_LFE_LABELS else -2)
else:
label_to_slot.append(int(slot))
if label in BED_LFE_LABELS:
direct_output_labels.append(3)
else:
desired = adm_xyz_to_render_coordinates(
*BED_CHANNEL_ADM_POSITION[label])
best = None
best_distance = float("inf")
for index, candidate in enumerate(BED_REFERENCE_POSITIONS_Q15):
if candidate == (-1, -1, -1):
continue
point = (candidate[0] / 32768.0, candidate[1] / 32768.0,
candidate[2] / 32768.0)
distance = math.dist(desired, point)
if distance < best_distance:
best = index
best_distance = distance
direct_output_labels.append(int(best))
return {
"output_labels": output_labels,
"label_to_slot": label_to_slot,
"direct_output_labels": direct_output_labels,
"automatic_attenuation_db": BED_LAYOUT_ATTENUATION_DB.get(
layout.name, 0.0),
}
# 标准 bed 布局名:由声明声道的集合唯一确定(9.1.6 这类常用叫法即由此得来)。 # 标准 bed 布局名:由声明声道的集合唯一确定(9.1.6 这类常用叫法即由此得来)。
BED_LAYOUT_NAMES = { BED_LAYOUT_NAMES = {
frozenset(("L", "R")): "2.0", frozenset(("L", "R")): "2.0",
@@ -115,6 +250,7 @@ BED_LAYOUT_NAMES = {
frozenset(("L", "R", "Ls", "Rs")): "4.0", frozenset(("L", "R", "Ls", "Rs")): "4.0",
frozenset(("L", "R", "C", "Ls", "Rs")): "5.0", frozenset(("L", "R", "C", "Ls", "Rs")): "5.0",
frozenset(("L", "R", "C", "LFE", "Ls", "Rs")): "5.1", frozenset(("L", "R", "C", "LFE", "Ls", "Rs")): "5.1",
frozenset(("L", "R", "Ls", "Rs", "Lrs", "Rrs")): "6.0",
frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Lrs", "Rrs")): "7.1", frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Lrs", "Rrs")): "7.1",
frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Lrs", "Rrs", "Lw", "Rw")): "9.1", frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Lrs", "Rrs", "Lw", "Rw")): "9.1",
frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Ltm", "Rtm")): "5.1.2", frozenset(("L", "R", "C", "LFE", "Ls", "Rs", "Ltm", "Rtm")): "5.1.2",
@@ -138,10 +274,14 @@ ADM_BED_712 = ("L", "R", "C", "LFE", "Ls", "Rs", "Lrs", "Rrs", "Ltf", "Rtf")
def adm_xyz_to_q(x, y, z): def adm_xyz_to_q(x, y, z):
"""ADM 笛卡尔坐标 → OAMD/渲染器使用的 Q15 方位三元组。""" """ADM 笛卡尔坐标 → OAMD/渲染器使用的 Q15 方位三元组。
q1 = int(round((float(x) + 1.0) * 0.5 * 32767.0))
q2 = int(round((1.0 - float(y)) * 0.5 * 32767.0)) Q15 满刻度是 32768(渲染器按 ``q / 32768`` 还原方位),因此用 32768 定标;
q3 = int(round(float(z) * 32767.0)) 除满刻度外按四舍五入取整。
"""
q1 = int(round((float(x) + 1.0) * 0.5 * 32768.0))
q2 = int(round((1.0 - float(y)) * 0.5 * 32768.0))
q3 = int(round(float(z) * 32768.0))
clamp = lambda value: max(0, min(32767, value)) # noqa: E731 clamp = lambda value: max(0, min(32767, value)) # noqa: E731
return clamp(q1), clamp(q2), clamp(q3) return clamp(q1), clamp(q2), clamp(q3)
@@ -196,7 +336,7 @@ def bed_layout(assignments):
def bed_object_positions(labels): def bed_object_positions(labels):
"""JOC 对象槽 → 固定 ADM 位置:(1..15) 对应未含 LFE 的 bed 声道。 """JOC 对象槽 → 固定 ADM 位置:(1..15) 对应未含 LFE 的 bed 声道。
positions 由声道标签唯一决定(Table B.10),与目标输出布局无关——同一个 bed 声道 positions 由声道标签唯一决定,与目标输出布局无关——同一个 bed 声道
在扬声器布局和双耳渲染里都落在同一个源位置。返回的字典只包含能给出固定位置的 在扬声器布局和双耳渲染里都落在同一个源位置。返回的字典只包含能给出固定位置的
标准声道;不是 bed 声道的对象不在其中,仍按 OAMD 坐标渲染。LFE 走独立通路, 标准声道;不是 bed 声道的对象不在其中,仍按 OAMD 坐标渲染。LFE 走独立通路,
不出现在对象槽里。 不出现在对象槽里。
+22
View File
@@ -188,3 +188,25 @@ class OamdPositionTimeline:
return np.stack( return np.stack(
[track.position_at(sample) for track in self.tracks], axis=0 [track.position_at(sample) for track in self.tracks], axis=0
).astype(np.float64, copy=False) ).astype(np.float64, copy=False)
class FixedPositionTimeline:
"""Static 15-object positions for synthesized objects (no OAMD payloads).
The multichannel path gives every input channel a fixed direction and never
submits metadata payloads, so the binaural adapter can query the same
``positions_at`` interface the OAMD timeline exposes.
"""
def __init__(self, positions):
array = np.asarray(positions, dtype=np.float64)
if array.shape != (15, 3):
raise ValueError(f"fixed positions must be (15,3), got {array.shape}")
if not np.all(np.isfinite(array)):
raise ValueError("fixed positions must be finite")
self._positions = array.copy()
self.payload_count = 0
self.transition_count = 0
def positions_at(self, sample):
del sample
return self._positions
+13 -7
View File
@@ -6,7 +6,7 @@ from pathlib import Path
import numpy as np import numpy as np
from binaural_metadata import OamdPositionTimeline from binaural_metadata import FixedPositionTimeline, OamdPositionTimeline
from public_filterbank import ANALYSIS_SYNTHESIS_LATENCY_SAMPLES from public_filterbank import ANALYSIS_SYNTHESIS_LATENCY_SAMPLES
@@ -48,7 +48,8 @@ class JochrtfBinauralRenderer:
object_delay_samples: int = 1473, object_delay_samples: int = 1473,
tail_seconds: float = 5.0, tail_seconds: float = 5.0,
chunk_frames: int = 64, chunk_frames: int = 64,
bed_channels=None): bed_channels=None,
fixed_positions=None):
required_interface = ( required_interface = (
"source_count", "default_profile", "set_source", "process", "source_count", "default_profile", "set_source", "process",
"finish", "finish_output_capacity", "info") "finish", "finish_output_capacity", "info")
@@ -79,9 +80,12 @@ class JochrtfBinauralRenderer:
from bed_layout import bed_object_positions from bed_layout import bed_object_positions
from sofa_metadata import SofaPositionTimeline from sofa_metadata import SofaPositionTimeline
timeline_type = SofaPositionTimeline if getattr(backend, "fixed_reference", False) else OamdPositionTimeline if fixed_positions is not None:
self.timeline = timeline_type( self.timeline = FixedPositionTimeline(fixed_positions)
15, bed_positions=bed_object_positions(bed_channels or ())) else:
timeline_type = SofaPositionTimeline if getattr(backend, "fixed_reference", False) else OamdPositionTimeline
self.timeline = timeline_type(
15, bed_positions=bed_object_positions(bed_channels or ()))
self._input_buffer = np.empty( self._input_buffer = np.empty(
(self.chunk_samples, SOURCE_CHANNELS), dtype=np.float64) (self.chunk_samples, SOURCE_CHANNELS), dtype=np.float64)
@@ -101,7 +105,8 @@ class JochrtfBinauralRenderer:
output_gain: float = 1.0, output_gain: float = 1.0,
chunk_frames: int = 64, chunk_frames: int = 64,
bed_channels=None, bed_channels=None,
room: bool = False) -> "JochrtfBinauralRenderer": room: bool = False,
fixed_positions=None) -> "JochrtfBinauralRenderer":
from jochrtf_backend import JochrtfBackend from jochrtf_backend import JochrtfBackend
source = resolve_compiled_hrtf_cache(cache) source = resolve_compiled_hrtf_cache(cache)
backend = JochrtfBackend.from_compiled_cache( backend = JochrtfBackend.from_compiled_cache(
@@ -116,7 +121,8 @@ class JochrtfBinauralRenderer:
object_delay_samples=object_delay_samples, object_delay_samples=object_delay_samples,
tail_seconds=tail_seconds, tail_seconds=tail_seconds,
chunk_frames=chunk_frames, chunk_frames=chunk_frames,
bed_channels=bed_channels) bed_channels=bed_channels,
fixed_positions=fixed_positions)
@property @property
def finish_capacity_samples(self) -> int: def finish_capacity_samples(self) -> int:
+1 -1
View File
@@ -115,7 +115,7 @@ def _parse_at(data, start_bit):
raise EmdfError(f"同一 EMDF 容器重复 payload id {payload_id}") raise EmdfError(f"同一 EMDF 容器重复 payload id {payload_id}")
has_sample_offset = bool(reader.read(1)) has_sample_offset = bool(reader.read(1))
sample_offset = (reader.read(12) >> 1) if has_sample_offset else 0 sample_offset = variable_bits(reader, 11) if has_sample_offset else 0
if reader.read(1): if reader.read(1):
variable_bits(reader, 11) # duration variable_bits(reader, 11) # duration
if reader.read(1): if reader.read(1):
+116 -26
View File
@@ -46,14 +46,28 @@ JOC_DENSE_OFFSET = {0: 48, 1: 96}
JOC_SPARSE_OFFSET = {0: 50, 1: 100} JOC_SPARSE_OFFSET = {0: 50, 1: 100}
class JocTruncatedPayload(ValueError):
"""载荷在语法读完之前用尽比特(例如 ID14 的 joc_data 被编码器写短)。
属于 ``ValueError`` 子类:现有把 ``IndexError``/``ValueError`` 当作“载荷不可用”
的调用点(容器候选打分等)不需要改动。
"""
class BR: class BR:
"""MSB-first 位读取器;位置以载荷内的 bit offset 表示。""" """MSB-first 位读取器;位置以载荷内的 bit offset 表示。
越界抛出 :class:`JocTruncatedPayload`,而不是让 bytes 下标抛 IndexError。
"""
def __init__(self, data, pos=0): def __init__(self, data, pos=0):
self.d = data self.d = data
self.p = pos self.p = pos
def bits(self, n): def bits(self, n):
if self.p + n > len(self.d) * 8:
raise JocTruncatedPayload(
f"位流越界: 需要 {n} bit @bit{self.p},载荷共 {len(self.d) * 8} bit")
v = 0 v = 0
for _ in range(n): for _ in range(n):
v = (v << 1) | ((self.d[self.p >> 3] >> (7 - (self.p & 7))) & 1) v = (v << 1) | ((self.d[self.p >> 3] >> (7 - (self.p & 7))) & 1)
@@ -79,54 +93,127 @@ def get_huff_code(mode, typ, nch):
else "joc_huff_code_fine_generic"] # MTX else "joc_huff_code_fine_generic"] # MTX
def parse_joc_header(payload): def _tail_info(out, br, payload):
"""记录 joc_data 结束位置与载荷尾部(剩余位数、尾字节)。"""
total_bits = len(payload) * 8
out["data_end_bits"] = br.p
out["remaining_bits"] = total_bits - br.p
out["tail_bytes"] = payload[br.p // 8:]
return total_bits
def parse_joc_header(payload, strict=True):
"""只读 joc_header 的 3+6+3 bit,不做 Huffman 解码。 """只读 joc_header 的 3+6+3 bit,不做 Huffman 解码。
跨层一致性校验(规格 §11.2)只需要这些字段,因此不必为校验再走一遍完整解析。 跨层一致性校验(规格 §11.2)只需要这些字段,因此不必为校验再走一遍完整解析。
配置不受支持时严格模式抛异常;宽松模式返回 ``status = 2``,由调用方按整帧无效处理。
""" """
br = BR(payload) br = BR(payload)
out = { out = {
"dmx_config_idx": br.bits(3), "dmx_config_idx": br.bits(3),
"num_objects_bits": br.bits(6), "num_objects_bits": br.bits(6),
"ext_config_idx": br.bits(3), "ext_config_idx": br.bits(3),
"status": 0,
} }
out["n_channels"] = JOC_NUM_CHANNELS.get(out["dmx_config_idx"]) out["n_channels"] = JOC_NUM_CHANNELS.get(out["dmx_config_idx"])
if out["n_channels"] is None: if out["n_channels"] is None:
raise ValueError(f"未知的 JOC downmix 配置 {out['dmx_config_idx']}") # 核心声道数决定对象语法的展开方式,配置未知时无法继续解析。
# 声明对象数超过 15 时截断到 15;截断本身另行记入 diagnostics。 if strict:
raise ValueError(f"未知的 JOC downmix 配置 {out['dmx_config_idx']}")
out["status"] = 2
out["objects_clamped"] = False
out["n_objects"] = 0
return out
# 声明对象数超过 15 时截断到 15;截断记入 diagnostics,整帧状态记为 3。
declared = out["num_objects_bits"] + 1 declared = out["num_objects_bits"] + 1
out["objects_clamped"] = declared > 15 out["objects_clamped"] = declared > 15
out["n_objects"] = min(declared, 15) out["n_objects"] = min(declared, 15)
if out["objects_clamped"]:
out["status"] = 3
return out return out
def parse_joc(payload): def parse_joc(payload):
"""解析 id14 载荷(joc() 位流)。返回字段 dict + 解析后剩余位数。""" """解析 id14 载荷(joc() 位流),返回字段 dict 与整帧状态 ``status``。
status:0 正常;2 downmix 配置不受支持;3 对象数超过 15 被截断;
4 参数带索引越界;5 offset_ts 非法;6 ext_config_idx=0 时对齐后仍有多余数据;
7 载荷在 joc_data 内提前结束(头与对象参数完整,只缺对象矩阵)。状态 7 是可
分类的坏帧:渲染器按无效帧保持上一帧矩阵,而不是中断播放。
"""
br = BR(payload) br = BR(payload)
out = parse_joc_header(payload) out = parse_joc_header(payload, strict=False)
n_objects, n_channels = out["n_objects"], out["n_channels"]
br.p = 12 br.p = 12
try:
return _parse_joc_body(payload, br, out)
except JocTruncatedPayload:
# 头(12 bit)之后的越界才可分类;头之前的越界在 parse_joc_header 里抛出,
# 那种载荷无法识别为 JOC 帧。
return _truncated_payload(payload, br, out)
def _truncated_payload(payload, br, out):
"""把 joc_data 内的越界归为整帧状态 7,保留已解析的头与对象参数。"""
out["status"] = 7
out.setdefault("clipgain_x_bits", 0)
out.setdefault("clipgain_y_bits", 0)
out.setdefault("clipgain", 1.0)
out.setdefault("seq_count_bits", 0)
objs = out.setdefault("objs", [])
while len(objs) < out.get("n_objects", 0):
objs.append({"present": 0})
for o in objs:
o.setdefault("present", 0)
if o["present"] and "n_bands" not in o:
# 参数区没读完的对象记为未出现,下游不会读到半成品字段。
o["present"] = 0
_tail_info(out, br, payload)
return out
def _parse_joc_body(payload, br, out):
"""从 clipgain 字段开始解析 joc() 载荷;比特越界抛 JocTruncatedPayload。"""
out["clipgain_x_bits"] = br.bits(3) out["clipgain_x_bits"] = br.bits(3)
out["clipgain_y_bits"] = br.bits(5) out["clipgain_y_bits"] = br.bits(5)
out["seq_count_bits"] = br.bits(10) out["seq_count_bits"] = br.bits(10)
# clipgain = 1 + (y/32)·2^(x−4),值域为 [1, 8.75]。 # clipgain = 2^(x−4)·(1 + y/32),值域为 [0.0625, 15.75]。
out["clipgain"] = 1 + out["clipgain_y_bits"] / 32.0 * 2 ** (out["clipgain_x_bits"] - 4) out["clipgain"] = 2 ** (out["clipgain_x_bits"] - 4) * (
1 + out["clipgain_y_bits"] / 32.0)
objs = [] objs = []
for obj in range(n_objects):
o = {}
o["present"] = br.bits(1)
if o["present"]:
o["num_bands_idx"] = br.bits(3)
o["n_bands"] = JOC_NUM_BANDS[o["num_bands_idx"]]
o["sparse"] = br.bits(1)
o["quant_idx"] = br.bits(1)
o["slope_idx"] = br.bits(1)
o["num_dpoints_bits"] = br.bits(1)
o["n_dpoints"] = o["num_dpoints_bits"] + 1
if o["slope_idx"] == 1:
o["offset_ts"] = [br.bits(5) + 1 for _ in range(o["n_dpoints"])]
objs.append(o)
out["objs"] = objs out["objs"] = objs
n_channels = out["n_channels"]
if n_channels is None:
# 核心声道数未知时对象语法无法展开,只保留头部字段与状态。
_tail_info(out, br, payload)
return out
for obj in range(out["n_objects"]):
o = {"present": br.bits(1)}
objs.append(o)
if not o["present"]:
continue
o["num_bands_idx"] = br.bits(3)
n_bands = JOC_NUM_BANDS.get(o["num_bands_idx"])
if n_bands is None:
# 参数带数未知时无法确定后续符号数量,停止解析。
if out["status"] == 0:
out["status"] = 4
_tail_info(out, br, payload)
return out
o["n_bands"] = n_bands
o["sparse"] = br.bits(1)
o["quant_idx"] = br.bits(1)
o["slope_idx"] = br.bits(1)
o["num_dpoints_bits"] = br.bits(1)
o["n_dpoints"] = o["num_dpoints_bits"] + 1
if o["slope_idx"] == 1:
offsets = [br.bits(5) for _ in range(o["n_dpoints"])]
o["offset_ts"] = offsets
invalid = any(offset >= 24 for offset in offsets)
if not invalid and o["n_dpoints"] > 1:
invalid = any(second <= first for first, second
in zip(offsets, offsets[1:]))
if invalid and out["status"] == 0:
out["status"] = 5
# joc_data(Huffman):dense 逐声道逐带读 MTX;sparse 读 IDX 后读 VEC。 # joc_data(Huffman):dense 逐声道逐带读 MTX;sparse 读 IDX 后读 VEC。
for obj, o in enumerate(objs): for obj, o in enumerate(objs):
if not o["present"]: if not o["present"]:
@@ -150,9 +237,12 @@ def parse_joc(payload):
o["mtx"].append(mtx) o["mtx"].append(mtx)
o["channel_idx"].append(None) o["channel_idx"].append(None)
o["vec"].append(None) o["vec"].append(None)
out["data_end_bits"] = br.p total_bits = _tail_info(out, br, payload)
out["remaining_bits"] = len(payload) * 8 - br.p if out["status"] == 0:
out["tail_bytes"] = payload[br.p // 8:] # 尾部先补 0 到 8 bit 对齐,补出来的位不是数据;对齐后仍有字节即多余。
pad = (8 - (br.p % 8)) % 8
if out["ext_config_idx"] == 0 and total_bits - br.p - pad > 0:
out["status"] = 6
return out return out
+23 -9
View File
@@ -16,12 +16,12 @@ QMF5_WINDOW = np.asarray(_TABLES["qmf5_window"], dtype=np.float64)
# 子带到参数带的映射表。 # 子带到参数带的映射表。
_TABLE39 = { _TABLE39 = {
23: "0 1 2 3 4 5 6 7 8 9 10 11 12 12 13 13 14 14 15 15 16 16 16 17 17 17 18 18 18 18 19 19 19 19 19 20 20 20 20 20 20 21 21 21 21 21 21 21 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22", 23: "0 1 2 3 4 5 6 7 8 9 10 11 12 12 13 13 14 14 15 15 16 16 16 17 17 17 18 18 18 18 19 19 19 19 19 20 20 20 20 20 20 21 21 21 21 21 21 21 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22 22",
15: "0 1 2 3 4 5 6 7 8 9 9 10 10 11 11 11 12 12 12 12 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14", 15: "0 1 2 3 4 5 6 7 8 9 9 10 10 10 11 11 11 11 12 12 12 12 12 13 13 13 13 13 13 13 13 13 13 13 13 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14 14",
12: "0 1 2 3 4 4 5 5 6 6 6 7 7 7 8 8 8 8 9 9 9 9 9 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11", 12: "0 1 2 3 4 4 5 5 6 6 6 7 7 7 8 8 8 8 9 9 9 9 9 10 10 10 10 10 10 10 10 10 10 10 10 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11 11",
9: "0 1 2 3 3 3 4 5 5 6 6 6 7 7 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8", 9: "0 1 2 3 3 4 4 5 5 6 6 6 6 6 7 7 7 7 7 7 7 7 7 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8 8",
7: "0 1 2 2 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6", 7: "0 1 2 2 3 3 3 3 4 4 4 4 4 4 5 5 5 5 5 5 5 5 5 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6 6",
5: "0 1 1 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4", 5: "0 1 1 2 2 2 2 2 2 3 3 3 3 3 3 3 3 3 3 3 3 3 3 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4 4",
3: "0 0 0 1 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2", 3: "0 0 0 1 1 1 1 1 1 1 1 1 1 1 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2 2",
1: "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0", 1: "0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0",
} }
_PB_MAP = {k: np.fromstring(v, dtype=np.int64, sep=" ") for k, v in _TABLE39.items()} _PB_MAP = {k: np.fromstring(v, dtype=np.int64, sep=" ") for k, v in _TABLE39.items()}
@@ -34,13 +34,26 @@ def sb_to_pb_table(n_bands):
raise ValueError(f"不支持的 JOC 参数带数: {n_bands}") from exc raise ValueError(f"不支持的 JOC 参数带数: {n_bands}") from exc
def interp_matrix(obj_info, dq, prev, n_ts=24): def interp_matrix(obj_info, dq, prev=None, n_ts=24):
"""按数据点和跨帧状态插值,返回 ``[channel,subband,timeslot]``。""" """按数据点和跨帧状态插值,返回 ``[channel,subband,timeslot]``。
``prev`` 是上一帧末时槽的矩阵;``prev`` 为 None 表示该对象还没有历史:
单数据点全部取 ``dp0``(硬切);双数据点前一半取 ``dp0``,后一半仍从 ``dp0``
插值到 ``dp1``。
"""
dq_sb = np.asarray(dq, dtype=np.float64)[:, :, sb_to_pb_table(dq.shape[2])] dq_sb = np.asarray(dq, dtype=np.float64)[:, :, sb_to_pb_table(dq.shape[2])]
previous = np.asarray(prev, dtype=np.float64) previous = None if prev is None else np.asarray(prev, dtype=np.float64)
n_dp = obj_info["n_dpoints"] n_dp = obj_info["n_dpoints"]
slope = obj_info["slope_idx"] slope = obj_info["slope_idx"]
if slope == 0: if slope == 0:
if previous is None:
if n_dp == 1:
return np.repeat(dq_sb[0, :, :, None], n_ts, axis=2)
half = n_ts // 2
a1 = (np.arange(n_ts - half, dtype=np.float64) + 1.0) / (n_ts - half)
first = np.repeat(dq_sb[0, :, :, None], half, axis=2)
second = dq_sb[0, :, :, None] * (1.0 - a1) + dq_sb[1, :, :, None] * a1
return np.concatenate((first, second), axis=2)
if n_dp == 1: if n_dp == 1:
alpha = (np.arange(n_ts, dtype=np.float64) + 1.0) / n_ts alpha = (np.arange(n_ts, dtype=np.float64) + 1.0) / n_ts
return previous[:, :, None] * (1.0 - alpha) + dq_sb[0, :, :, None] * alpha return previous[:, :, None] * (1.0 - alpha) + dq_sb[0, :, :, None] * alpha
@@ -53,9 +66,10 @@ def interp_matrix(obj_info, dq, prev, n_ts=24):
ts = np.arange(n_ts) ts = np.arange(n_ts)
offsets = obj_info.get("offset_ts", []) offsets = obj_info.get("offset_ts", [])
base = dq_sb[0, :, :, None] if previous is None else previous[:, :, None]
if n_dp == 1: if n_dp == 1:
return np.where(ts[None, None, :] < offsets[0], previous[:, :, None], dq_sb[0, :, :, None]) return np.where(ts[None, None, :] < offsets[0], base, dq_sb[0, :, :, None])
out = np.where(ts[None, None, :] < offsets[0], previous[:, :, None], dq_sb[0, :, :, None]) out = np.where(ts[None, None, :] < offsets[0], base, dq_sb[0, :, :, None])
return np.where(ts[None, None, :] < offsets[1], out, dq_sb[1, :, :, None]) return np.where(ts[None, None, :] < offsets[1], out, dq_sb[1, :, :, None])
+16 -4
View File
@@ -25,14 +25,16 @@ def _joc_pair_score(container):
(ID14 声明的对象数超出其字节所能容纳的范围、ID11 未声明 groupid)既不是一对 (ID14 声明的对象数超出其字节所能容纳的范围、ID11 未声明 groupid)既不是一对
合法载荷,也不可能是本帧的 JOC 参数。 合法载荷,也不可能是本帧的 JOC 参数。
2 = 两个载荷 groupid 相同且 ID14 完整解析;1 = ID14 完整解析;0 = 其余。 2 = 两个载荷 groupid 相同且 ID14 完整解析;1 = ID14 完整解析;0 = 其余,
包括在 joc_data 内提前结束的载荷(状态 7):它缺对象矩阵,不应赢得容器选择。
""" """
payload = container.payloads.get(14) payload = container.payloads.get(14)
if payload is None or 11 not in container.payloads: if payload is None or 11 not in container.payloads:
return 0 return 0
parses = True parses = False
try: try:
parse_joc(payload) parsed = parse_joc(payload)
parses = parsed["status"] not in (2, 4, 7)
except (IndexError, ValueError): except (IndexError, ValueError):
parses = False parses = False
group_ids = getattr(container, "group_ids", None) or {} group_ids = getattr(container, "group_ids", None) or {}
@@ -297,7 +299,14 @@ class DirectPayloadIndex:
continue continue
if 14 in container.payloads: if 14 in container.payloads:
parsed = parse_joc_header(container.payloads[14]) parsed = parse_joc_header(container.payloads[14], strict=False)
if parsed["n_channels"] is None:
raise UnsupportedVariantError(
"joc", "downmix_config_unsupported",
"ID14 的 joc_dmx_config_idx 不受支持(合法值 0..4)",
frame=frame_number,
details={"dmx_config_idx": parsed["dmx_config_idx"],
"payload": bytes_descriptor(container.payloads[14])})
parsed["payload"] = container.payloads[14] parsed["payload"] = container.payloads[14]
frame_bed = (program_bed_summary(container.payloads[11]) frame_bed = (program_bed_summary(container.payloads[11])
if 11 in container.payloads else None) if 11 in container.payloads else None)
@@ -449,6 +458,7 @@ def inspect(index, limit=None, print_frames=False):
active = Counter() active = Counter()
band_totals = Counter() band_totals = Counter()
sparse_counts = Counter() sparse_counts = Counter()
statuses = Counter()
clipgains = [] clipgains = []
for seq, row in enumerate(rows): for seq, row in enumerate(rows):
frame_number = int(row.get("frame", seq)) frame_number = int(row.get("frame", seq))
@@ -541,6 +551,7 @@ def inspect(index, limit=None, print_frames=False):
active[len(rec["objects_present"])] += 1 active[len(rec["objects_present"])] += 1
band_totals.update({int(k): v for k, v in rec["parameter_bands"].items()}) band_totals.update({int(k): v for k, v in rec["parameter_bands"].items()})
sparse_counts[len(rec["sparse_objects"])] += 1 sparse_counts[len(rec["sparse_objects"])] += 1
statuses[parsed["status"]] += 1
clipgains.append(float(parsed["clipgain"])) clipgains.append(float(parsed["clipgain"]))
if print_frames: if print_frames:
print(json.dumps(rec, ensure_ascii=False, separators=(",", ":"))) print(json.dumps(rec, ensure_ascii=False, separators=(",", ":")))
@@ -560,6 +571,7 @@ def inspect(index, limit=None, print_frames=False):
"objects": k[3], "frames": count} "objects": k[3], "frames": count}
for k, count in sorted(configs.items()) for k, count in sorted(configs.items())
], ],
"frame_status_histogram": {str(k): v for k, v in sorted(statuses.items())},
"active_object_count_histogram": {str(k): v for k, v in sorted(active.items())}, "active_object_count_histogram": {str(k): v for k, v in sorted(active.items())},
"parameter_band_totals": {str(k): v for k, v in sorted(band_totals.items())}, "parameter_band_totals": {str(k): v for k, v in sorted(band_totals.items())},
"sparse_object_count_histogram": {str(k): v for k, v in sorted(sparse_counts.items())}, "sparse_object_count_histogram": {str(k): v for k, v in sorted(sparse_counts.items())},
+285
View File
@@ -0,0 +1,285 @@
"""Render ordinary 2.0 / 5.1 / 7.1 PCM WAV to binaural audio.
The binaural renderer is a JOC-object renderer: it expects sixteen source
channels (LFE plus fifteen objects) with a direction per object. For ordinary
multichannel PCM there is no OAMD, so this module
* maps the layout's channel order to fixed ADM directions,
* turns each non-LFE channel into one synthetic object and the LFE channel
into source 0,
* feeds the same Rosella / jochrtf binaural adapter the JOC path uses.
The HRTF runtime is native 48 kHz. Prefer letting ffmpeg/libsoxr produce a
48 kHz float32 WAV and feed that in directly:
ffmpeg -i INPUT -af "aresample=resampler=soxr:precision=28:osr=48000" \
-c:a pcm_f32le OUT.wav
The module then renders and writes 48 kHz stereo with no further sample-rate
conversion. The supported families are 44.1k (44.1/88.2/176.4) and 48k
(48/96/192), up to 192 kHz; a non-48 kHz input is converted once on the way in,
and the output stays 48 kHz by default. An explicit
non-48k output rate is possible but adds a second conversion.
"""
from __future__ import annotations
import argparse
import math
import sys
from pathlib import Path
import numpy as np
from scipy import signal
from scipy.io import wavfile
from binaural_renderer import JochrtfBinauralRenderer
from rosella_binaural_renderer import (DEFAULT_PERSONALIZED_HEADPHONE,
RosellaBinauralRenderer)
from rosella_bank import DEFAULT_FILTERBANK
TARGET_RATE = 48000
FRAME_SAMPLES = 1536
SOURCE_CHANNELS = 16
# WAV channel order follows the ffmpeg layout definitions:
# 2.0 = FL FR
# 5.1 = FL FR FC LFE BL BR
# 7.1 = FL FR FC LFE BL BR SL SR
# BL/BR are the rear pair, SL/SR the side pair.
LAYOUTS = {
"2.0": (
("L", (-1.0, 1.0, 0.0)),
("R", (1.0, 1.0, 0.0)),
),
"5.1": (
("L", (-1.0, 1.0, 0.0)),
("R", (1.0, 1.0, 0.0)),
("C", (0.0, 1.0, 0.0)),
("LFE", None),
("BL", (-1.0, -1.0, 0.0)),
("BR", (1.0, -1.0, 0.0)),
),
"7.1": (
("L", (-1.0, 1.0, 0.0)),
("R", (1.0, 1.0, 0.0)),
("C", (0.0, 1.0, 0.0)),
("LFE", None),
("BL", (-1.0, -1.0, 0.0)),
("BR", (1.0, -1.0, 0.0)),
("SL", (-1.0, 0.0, 0.0)),
("SR", (1.0, 0.0, 0.0)),
),
}
LAYOUT_CHANNELS = {"2.0": 2, "5.1": 6, "7.1": 8}
def detect_layout(channels: int) -> str:
for name, count in LAYOUT_CHANNELS.items():
if count == int(channels):
return name
raise ValueError(
f"cannot infer a binaural layout for {channels} channels; "
f"supported layouts: 2.0 (2), 5.1 (6), 7.1 (8)")
def object_positions(layout: str) -> np.ndarray:
"""Return the 15 synthetic object directions for one layout."""
if layout not in LAYOUTS:
raise ValueError(f"unknown layout {layout!r}")
positions = np.zeros((15, 3), dtype=np.float64)
slot = 0
for label, position in LAYOUTS[layout]:
if position is None:
continue
if slot >= 15:
raise ValueError(f"layout {layout} needs more than 15 object slots")
positions[slot] = position
slot += 1
return positions
def pcm_to_objects(pcm: np.ndarray, layout: str) -> np.ndarray:
"""Map interleaved layout PCM ``[samples, channels]`` to 16 JOC sources."""
pcm = np.asarray(pcm, dtype=np.float64)
if pcm.ndim != 2 or pcm.shape[1] != LAYOUT_CHANNELS[layout]:
raise ValueError(
f"{layout} expects {LAYOUT_CHANNELS[layout]} channels, got {pcm.shape}")
objects = np.zeros((len(pcm), SOURCE_CHANNELS), dtype=np.float64)
slot = 0
for channel, (label, position) in enumerate(LAYOUTS[layout]):
if position is None:
if label != "LFE":
raise ValueError(f"only LFE may omit a direction, got {label}")
objects[:, 0] = pcm[:, channel]
continue
objects[:, 1 + slot] = pcm[:, channel]
slot += 1
return objects
def _rate_ratio(source_rate: int, target_rate: int) -> tuple[int, int]:
common = math.gcd(int(source_rate), int(target_rate))
return int(target_rate) // common, int(source_rate) // common
def resample(data: np.ndarray, source_rate: int, target_rate: int) -> np.ndarray:
"""Resample ``[samples, channels]`` with a polyphase anti-alias filter."""
if int(source_rate) == int(target_rate):
return np.asarray(data, dtype=np.float64)
if int(source_rate) <= 0 or int(target_rate) <= 0:
raise ValueError("sample rates must be positive")
up, down = _rate_ratio(source_rate, target_rate)
result = signal.resample_poly(
np.asarray(data, dtype=np.float64), up, down, axis=0,
window=("kaiser", 10.0))
expected = int(round(len(data) * float(target_rate) / float(source_rate)))
if len(result) > expected:
result = result[:expected]
elif len(result) < expected:
pad = np.zeros((expected - len(result), result.shape[1]), dtype=result.dtype)
result = np.concatenate((result, pad), axis=0)
return result
def read_wav(path: str | Path) -> tuple[int, np.ndarray]:
rate, data = wavfile.read(str(path))
if data.ndim == 1:
data = data[:, None]
if np.issubdtype(data.dtype, np.integer):
scale = float(np.iinfo(data.dtype).max)
data = data.astype(np.float64) / scale
else:
data = data.astype(np.float64)
return int(rate), data
def write_wav(path: str | Path, rate: int, data: np.ndarray) -> None:
target = Path(path)
target.parent.mkdir(parents=True, exist_ok=True)
wavfile.write(str(target), int(rate), np.asarray(data, dtype=np.float32))
def _make_renderer(*, backend: str, model, compiled_hrtf, mode: str,
filterbank: int, tail_seconds: float,
fixed_positions: np.ndarray):
if backend == "rosella":
return RosellaBinauralRenderer(
model, mode=mode, filterbank=filterbank,
tail_seconds=tail_seconds, fixed_positions=fixed_positions)
if backend == "jochrtf":
if not compiled_hrtf:
raise ValueError("--backend jochrtf requires --compiled-hrtf PATH")
return JochrtfBinauralRenderer.from_compiled_cache(
compiled_hrtf, mode=mode, tail_seconds=tail_seconds,
fixed_positions=fixed_positions)
raise ValueError(f"unknown binaural backend {backend!r}")
def render_file(input_path, output_path, *, layout="auto",
backend="rosella", model=None, compiled_hrtf=None,
mode="mid", filterbank: int = DEFAULT_FILTERBANK,
tail_seconds: float = 2.0, output_rate=48000) -> dict:
"""Decode a PCM WAV, synthesize objects and render it binaurally."""
input_rate, pcm = read_wav(input_path)
channels = pcm.shape[1]
resolved = detect_layout(channels) if layout == "auto" else layout
if resolved not in LAYOUTS:
raise ValueError(f"unsupported layout {resolved!r}")
if channels != LAYOUT_CHANNELS[resolved]:
raise ValueError(
f"layout {resolved} expects {LAYOUT_CHANNELS[resolved]} channels, "
f"WAV has {channels}")
input_resampled = input_rate != TARGET_RATE
internal = pcm if not input_resampled else resample(
pcm, input_rate, TARGET_RATE)
objects = pcm_to_objects(internal, resolved)
padded = np.zeros(
((len(objects) + FRAME_SAMPLES - 1) // FRAME_SAMPLES * FRAME_SAMPLES,
SOURCE_CHANNELS), dtype=np.float64)
padded[:len(objects)] = objects
positions = object_positions(resolved)
renderer = _make_renderer(
backend=backend, model=model, compiled_hrtf=compiled_hrtf, mode=mode,
filterbank=filterbank, tail_seconds=tail_seconds,
fixed_positions=positions)
chunks = []
for start in range(0, len(padded), FRAME_SAMPLES):
chunks.append(renderer.render_frame(padded[start:start + FRAME_SAMPLES]))
chunks.append(renderer.finish())
stereo = np.concatenate([chunk for chunk in chunks if len(chunk)], axis=0)
if output_rate in (48000, "48000", "48k"):
target_rate = TARGET_RATE
elif output_rate == "input":
target_rate = input_rate
else:
target_rate = int(output_rate)
if target_rate != TARGET_RATE:
stereo = resample(stereo, TARGET_RATE, target_rate)
write_wav(output_path, target_rate, stereo)
return {
"input": str(Path(input_path).resolve()),
"output": str(Path(output_path).resolve()),
"layout": resolved,
"input_channels": channels,
"input_rate": input_rate,
"input_resampled": input_resampled,
"render_rate": TARGET_RATE,
"output_rate": target_rate,
"input_samples": len(pcm),
"output_samples": len(stereo),
"backend": backend,
"mode": mode,
}
def main(argv=None) -> int:
parser = argparse.ArgumentParser(
description="Render 2.0/5.1/7.1 PCM WAV files binaurally")
parser.add_argument("input", type=Path, help="interleaved PCM WAV")
parser.add_argument("-o", "--output", type=Path)
parser.add_argument("--layout", default="auto",
choices=("auto", "2.0", "5.1", "7.1"))
parser.add_argument("--backend", default="rosella",
choices=("rosella", "jochrtf"))
parser.add_argument("--model", type=Path, default=None,
help="Rosella .personalized_headphone model")
parser.add_argument("--compiled-hrtf", type=Path, default=None,
help="JOC .jochrtf compiled cache")
parser.add_argument("--binaural-mode", default="mid",
choices=("near", "mid", "far"))
parser.add_argument("--filterbank", type=int, default=DEFAULT_FILTERBANK)
parser.add_argument("--tail-seconds", type=float, default=2.0)
parser.add_argument("--output-rate", default="48000",
help="48k (default, native HRTF rate) or an explicit rate; "
"non-48k output adds a second resampling pass")
parser.add_argument("--list-layouts", action="store_true")
args = parser.parse_args(argv)
if args.list_layouts:
for name, channels in LAYOUT_CHANNELS.items():
print(f"{name}: {channels} channels")
return 0
output = args.output or Path(str(args.input) + ".binaural.wav")
model = args.model or DEFAULT_PERSONALIZED_HEADPHONE
info = render_file(
args.input, output, layout=args.layout, backend=args.backend,
model=model, compiled_hrtf=args.compiled_hrtf,
mode=args.binaural_mode, filterbank=args.filterbank,
tail_seconds=args.tail_seconds, output_rate=args.output_rate)
for key, value in info.items():
print(f"{key}: {value}")
if info["input_resampled"]:
print(
"warning: input was not 48 kHz and was converted once internally; "
"prefer ffmpeg/libsoxr to produce a 48 kHz float32 WAV first, e.g. "
"ffmpeg -i INPUT -af \"aresample=resampler=soxr:precision=28:osr=48000\" "
"-c:a pcm_f32le OUT.wav",
file=sys.stderr)
return 0
if __name__ == "__main__":
sys.exit(main())
+624 -152
View File
@@ -5,6 +5,8 @@ ch1..15;bed/ISF/未激活对象没有位置字段,保持上一帧位置。el
驱动,未知 element 按边界跳过;个别编码器的 ``oa_element_size`` 比实际内容短时以 驱动,未知 element 按边界跳过;个别编码器的 ``oa_element_size`` 比实际内容短时以
结构解析为准,差异记入 ``diagnostics``,不算变体错误。 结构解析为准,差异记入 ``diagnostics``,不算变体错误。
""" """
import math
import numpy as np import numpy as np
from variant_error import UnsupportedVariantError, bytes_descriptor from variant_error import UnsupportedVariantError, bytes_descriptor
@@ -18,6 +20,25 @@ RAMP_DURATION_INDEX = (
1024, 1600, 1601, 1602, 1920, 2000, 2002, 2048, 1024, 1600, 1601, 1602, 1920, 2000, 2002, 2048,
) )
OBJECT_ELEMENT_ID = 1 OBJECT_ELEMENT_ID = 1
TRIM_ELEMENT_ID = 2
# Table 12 of ETSI TS 103 420:trim 电平表,4 bit 索引,单位 dB。
# 索引 15 的 -36 dB 是最小值(并非静音)。
TRIM_LUT = (6.0, 3.0, 1.5, 0.75, -0.75, -1.5, -3.0, -4.5,
-6.0, -7.5, -9.0, -10.5, -12.0, -13.5, -15.0, -36.0)
# bal3d_y 的编码为 (code + 1) / 16,幅度落在 [1/16, 1],取不到 0。
BALANCE_STEP = 16
# 5.5.3/5.5.5 divergence 码表(公开实现固定值),Q15 量纲。
DIVERGENCE_TABLE = (
0, 0, 66, 117, 209, 331, 467, 659,
829, 1042, 1309, 1643, 2059, 2302, 2573, 2872,
3202, 3564, 3960, 4391, 4856, 5354, 5882, 6437,
7014, 7605, 8204, 8803, 9394, 9970, 10524, 11051,
11548, 12011, 12441, 12835, 13197, 13525, 13823, 14092,
14335, 14553, 14748, 14924, 15081, 15221, 15347, 15459,
15559, 15649, 15728, 15799, 15863, 15970, 16055, 16123,
16176, 16219, 16267, 16301, 16332, 16355, 16368, 16384,
)
DIVERGENCE_QUICK_INDEX = (26, 29, 32, 63)
# 5.6.0.11 Table 11b:ISF 类型 → 对象数。 # 5.6.0.11 Table 11b:ISF 类型 → 对象数。
ISF_OBJECT_COUNTS = (4, 8, 10, 14, 15, 30) ISF_OBJECT_COUNTS = (4, 8, 10, 14, 15, 30)
# 5.6.1.1.4 Table 12:10 bit 标准 bed 掩码。array position 9(RC_L/RC_R)是 channel # 5.6.1.1.4 Table 12:10 bit 标准 bed 掩码。array position 9(RC_L/RC_R)是 channel
@@ -172,6 +193,8 @@ def _parse_program_assignment(reader, raw_payload):
"content_description": None, "content_description": None,
"bed_assignments": [], "bed_assignments": [],
"num_bed_objects": 0, "num_bed_objects": 0,
"b_bed_chan_distribute": False,
"bed_mask": 0,
"isf_idx": None, "isf_idx": None,
"num_isf_objects": 0, "num_isf_objects": 0,
"num_dynamic_objects": None, "num_dynamic_objects": None,
@@ -180,6 +203,11 @@ def _parse_program_assignment(reader, raw_payload):
program["lfe_present"] = bool(reader.read(1)) program["lfe_present"] = bool(reader.read(1))
# 5.6.4.8:对象顺序为 bed → ISF → dynamic,LFE 属于 bed,排在最前。 # 5.6.4.8:对象顺序为 bed → ISF → dynamic,LFE 属于 bed,排在最前。
program["num_bed_objects"] = 1 if program["lfe_present"] else 0 program["num_bed_objects"] = 1 if program["lfe_present"] else 0
if program["lfe_present"]:
program["bed_assignments"] = [{
"instance": 0, "lfe_only": True, "channels": ["RC_LFE"],
}]
program["bed_mask"] = 0x8
return program return program
mask = reader.read(4) mask = reader.read(4)
@@ -200,16 +228,19 @@ def _parse_program_assignment(reader, raw_payload):
continue continue
if reader.read(1): # b_standard_chan_assign if reader.read(1): # b_standard_chan_assign
bed_mask = reader.read(10) bed_mask = reader.read(10)
program["bed_assignments"].append({ standard = True
"instance": instance, "lfe_only": False, "standard": True, channels = standard_bed_channels(bed_mask)
"mask": bed_mask, "channels": standard_bed_channels(bed_mask), wide_mask = standard_bed_mask(bed_mask)
})
else: else:
bed_mask = reader.read(17) bed_mask = reader.read(17)
program["bed_assignments"].append({ standard = False
"instance": instance, "lfe_only": False, "standard": False, channels = nonstandard_bed_channels(bed_mask)
"mask": bed_mask, "channels": nonstandard_bed_channels(bed_mask), wide_mask = bed_mask
}) program["bed_assignments"].append({
"instance": instance, "lfe_only": False, "standard": standard,
"mask": bed_mask, "channels": channels,
})
program["bed_mask"] |= wide_mask
program["num_bed_objects"] = sum( program["num_bed_objects"] = sum(
len(assignment["channels"]) for assignment in program["bed_assignments"]) len(assignment["channels"]) for assignment in program["bed_assignments"])
if mask & 0x2: if mask & 0x2:
@@ -228,82 +259,264 @@ def _parse_program_assignment(reader, raw_payload):
return program return program
def _parse_object_info_block(reader, object_index, in_bed_or_isf, raw_payload, retain_spatial=False): # 17 bit 非标准 bed 标签在 wide mask 中的位顺序(与 NONSTD_BED_LABELS 相同)。
"""5.5.9 ``object_info_block(0)``:一个对象的属性更新。""" _STANDARD_BED_TO_WIDE = (0x3, 0x4, 0x8, 0x30, 0xC0,
0x300, 0xC00, 0x3000, 0xC000, 0x10000)
def standard_bed_mask(bed_mask):
"""5.6.1.1.4 的 10 bit 标准掩码 → 17 bit 非标准 bed 掩码。"""
wide = 0
for bit, expanded in enumerate(_STANDARD_BED_TO_WIDE):
if (int(bed_mask) >> bit) & 1:
wide |= expanded
return wide
def _q15_ratio(numerator, denominator):
"""Q15 of ``numerator / denominator``, rounded half up and capped."""
return min(32767, int(math.floor(32768.0 * numerator / denominator + 0.5)))
def _gain_q(gain_idx, gain_bits):
"""gain_idx/bits → 整数增益 Qg(单位增益 4096)。"""
if gain_idx == 0:
return 4096
if gain_idx == 1:
return 0
if gain_idx == 2:
bits = int(gain_bits)
db = 15 - bits if bits < 15 else 14 - bits
return int(math.floor(4096.0 * (10.0 ** (db / 20.0))))
raise ValueError("gain_idx=3 需要同块前一对象的状态")
def _size_q(mode, values):
"""size_idx 与 5 bit 数值 → (sx, sy, sz) 整数 Q15。"""
if mode == 1:
value = _q15_ratio(int(values), 31)
return (value, value, value)
if mode == 2:
return tuple(_q15_ratio(int(value), 31) for value in values)
return (0, 0, 0)
_RENDER_DEFAULTS = {
"qx": 16384, "qy": 16384, "qz": 0,
"coarse": (31, 31, 0),
"zone_idx": 0, "enable_elevation": True,
"size_mode": 0, "size_q": (0, 0, 0), "snap": 0,
"divergence_q": 0,
}
def _clamp_coarse(coarse):
x, y, z = coarse
return (min(max(int(x), 0), 62), min(max(int(y), 0), 62),
min(max(int(z), -15), 15))
def _coarse_to_q15(coarse):
x, y, z = coarse
qx = _q15_ratio(int(x), 62)
qy = _q15_ratio(int(y), 62)
magnitude = _q15_ratio(abs(int(z)), 15)
return qx, qy, (-magnitude if z < 0 else magnitude)
_POSITION_DELTA = (0, 1, 2, 3, -4, -3, -2, -1)
_REFINE_DELTA = (1, 2, -1, -2)
def _refine_xy(index):
"""Q15 code for one refined X/Y position index ``5*coarse + delta + 2``."""
return min(32767, max(0, int(math.floor(
(int(index) - 2) * 32768.0 / 310.0 + 0.5))))
def _refine_z(index):
"""Q15 code for one refined Z position index (signed)."""
return min(32767, max(-32767, int(math.floor(
(int(index) - 2) * 32768.0 / 75.0 + 0.5))))
def _copy_render_state(previous):
state = {key: previous[key] for key in _RENDER_DEFAULTS}
state["coarse"] = tuple(previous["coarse"])
state["size_q"] = tuple(previous["size_q"])
state["qx"], state["qy"], state["qz"] = previous["qx"], previous["qy"], previous["qz"]
state["divergence_q"] = int(previous["divergence_q"])
return state
def parse_object_info_block(reader, object_index, block_index, kind,
previous_block, raw_payload, retain_spatial=False,
object_states=None):
"""5.5.9 ``object_info_block``:展开一个对象的一个块。"""
info = { info = {
"object": object_index, "object": object_index,
"in_bed_or_isf": bool(in_bed_or_isf), "block": block_index,
"kind": kind,
"in_bed_or_isf": kind != "dynamic",
"not_active": bool(reader.read(1)), "not_active": bool(reader.read(1)),
} }
# blk == 0 且对象激活时 basic info 恒为 full update(status 0b01)。 inactive = info["not_active"]
basic_status = 0 if info["not_active"] else 1 if inactive:
info["basic_status"] = basic_status info["basic_status"] = 0
if basic_status in (1, 3): info["gain_q"] = 0
if basic_status == 1: info["priority_q"] = 0
# 5.5.10:object_basic_info[] = {true, true}; render = dict(_RENDER_DEFAULTS)
# 5.6.4.12:array[1] = object_gain_idx,array[0] = b_default_object_priority。 render["coarse"] = (31, 31, 0)
gain_present = priority_present = True render["size_q"] = tuple(render["size_q"])
else: info.update({key: render[key] for key in render if key != "coarse"})
flags = reader.read(2) info["coarse"] = render["coarse"]
gain_present = bool(flags & 1) info["render_status"] = 0
priority_present = bool(flags & 2) info["qx"], info["qy"], info["qz"] = _coarse_to_q15(render["coarse"])
if gain_present: else:
basic_status = 1 if block_index == 0 else reader.read(2)
info["basic_status"] = basic_status
if basic_status == 0:
info["gain_q"] = 0
info["priority_q"] = 0
elif basic_status == 1:
gain_idx = reader.read(2) gain_idx = reader.read(2)
info["gain_idx"] = gain_idx info["gain_idx"] = gain_idx
if gain_idx == 2: gain_bits = reader.read(6) if gain_idx == 2 else None
info["gain_bits"] = reader.read(6) if gain_bits is not None:
if priority_present: info["gain_bits"] = gain_bits
if gain_idx == 3:
if object_index == 0:
info["gain_q"] = 4096
elif object_states[object_index - 1][block_index]["not_active"]:
info["gain_q"] = 0
else:
info["gain_q"] = object_states[object_index - 1][block_index]["gain_q"]
else:
info["gain_q"] = _gain_q(gain_idx, gain_bits)
default_priority = bool(reader.read(1)) default_priority = bool(reader.read(1))
info["default_priority"] = default_priority info["default_priority"] = default_priority
if not default_priority: if default_priority:
info["priority_bits"] = reader.read(5) info["priority_q"] = 32767
render_status = 0 if (info["not_active"] or info["in_bed_or_isf"]) else 1 else:
info["render_status"] = render_status bits = reader.read(5)
if render_status in (1, 3): info["priority_bits"] = bits
if render_status == 1: info["priority_q"] = 1024 * bits
# 5.5.11:obj_render_info[] = {true, true, true, true}。 elif basic_status == 2:
position_present = zone_present = size_present = screen_present = True info["gain_q"] = previous_block["gain_q"]
info["priority_q"] = previous_block["priority_q"]
else: else:
flags = reader.read(4) flags = reader.read(2)
position_present = bool(flags & 0x1) info["basic_mask"] = flags
zone_present = bool(flags & 0x2) info["gain_q"] = previous_block["gain_q"]
size_present = bool(flags & 0x4) info["priority_q"] = previous_block["priority_q"]
screen_present = bool(flags & 0x8) if flags & 1:
if position_present: gain_idx = reader.read(2)
# blk == 0 时 b_differential_position_specified 恒为 FALSE。 info["gain_idx"] = gain_idx
x = reader.read(6) gain_bits = reader.read(6) if gain_idx == 2 else None
y = reader.read(6) if gain_bits is not None:
z_sign = reader.read(1) info["gain_bits"] = gain_bits
z = reader.read(4) if gain_idx == 3:
info["position"] = (x, y, z if z_sign else -z) if object_index == 0:
outside = bool(reader.read(1)) info["gain_q"] = 4096
infinite = False elif object_states[object_index - 1][block_index]["not_active"]:
distance_index = None info["gain_q"] = 0
if outside: else:
infinite = bool(reader.read(1)) info["gain_q"] = object_states[object_index - 1][block_index]["gain_q"]
if not infinite: else:
distance_index = reader.read(4) info["gain_q"] = _gain_q(gain_idx, gain_bits)
if retain_spatial: if flags & 2:
info.update(distance_specified=outside, at_infinity=infinite, default_priority = bool(reader.read(1))
distance_factor_idx=distance_index) info["default_priority"] = default_priority
if zone_present: if default_priority:
info["zone_constraints_idx"] = reader.read(3) info["priority_q"] = 32767
info["enable_elevation"] = bool(reader.read(1)) else:
if size_present: bits = reader.read(5)
size_idx = reader.read(2) info["priority_bits"] = bits
info["object_size_idx"] = size_idx info["priority_q"] = 1024 * bits
if size_idx == 1:
reader.read(5) render_status = 0 if kind in ("bed", "isf") else (
elif size_idx == 2: 1 if block_index == 0 else reader.read(2))
reader.read(15) info["render_status"] = render_status
if screen_present: if render_status in (1, 3):
screen_ref = bool(reader.read(1)) if render_status == 1:
screen_bits = reader.read(3) if screen_ref else None position_present = zone_present = size_present = screen_present = True
depth_index = reader.read(2) if screen_ref else None flags = 0xF
if retain_spatial: else:
info.update(use_screen_ref=screen_ref, screen_factor_bits=screen_bits, flags = reader.read(4)
depth_factor_idx=depth_index) position_present = bool(flags & 0x1)
info["snap"] = bool(reader.read(1)) zone_present = bool(flags & 0x2)
size_present = bool(flags & 0x4)
screen_present = bool(flags & 0x8)
info["render_mask"] = flags
state = _copy_render_state(previous_block) if render_status == 3 \
else dict(_RENDER_DEFAULTS)
state["coarse"] = tuple(state["coarse"])
state["size_q"] = tuple(state["size_q"])
if position_present:
differential = (block_index > 0 and bool(reader.read(1)))
if differential:
dx, dy, dz = (_POSITION_DELTA[reader.read(3)]
for _axis in range(3))
coarse = (previous_block["coarse"][0] + dx,
previous_block["coarse"][1] + dy,
previous_block["coarse"][2] + dz)
else:
x = reader.read(6)
y = reader.read(6)
z_sign = reader.read(1)
z = reader.read(4)
coarse = (x, y, z if z_sign else -z)
state["coarse"] = _clamp_coarse(coarse)
outside = bool(reader.read(1))
infinite = False
distance_index = None
if outside:
infinite = bool(reader.read(1))
if not infinite:
distance_index = reader.read(4)
if retain_spatial:
info["distance_specified"] = outside
info["at_infinity"] = infinite
info["distance_factor_idx"] = distance_index
if zone_present:
state["zone_idx"] = reader.read(3)
state["enable_elevation"] = bool(reader.read(1))
if size_present:
size_idx = reader.read(2)
state["size_mode"] = size_idx
if size_idx == 1:
state["size_q"] = _size_q(1, reader.read(5))
elif size_idx == 2:
state["size_q"] = _size_q(2, (reader.read(5), reader.read(5),
reader.read(5)))
else:
state["size_q"] = (0, 0, 0)
if screen_present:
screen_ref = bool(reader.read(1))
screen_bits = reader.read(3) if screen_ref else None
depth_index = reader.read(2) if screen_ref else None
if retain_spatial:
info["use_screen_ref"] = screen_ref
info["screen_factor_bits"] = screen_bits
info["depth_factor_idx"] = depth_index
state["snap"] = 1 if reader.read(1) else 0
info.update({key: state[key] for key in state if key != "coarse"})
info["coarse"] = state["coarse"]
info["qx"], info["qy"], info["qz"] = _coarse_to_q15(state["coarse"])
elif render_status == 2:
info.update({key: previous_block[key] for key in _RENDER_DEFAULTS
if key != "coarse"})
info["coarse"] = tuple(previous_block["coarse"])
info["size_q"] = tuple(previous_block["size_q"])
info["qx"], info["qy"], info["qz"] = (
previous_block["qx"], previous_block["qy"], previous_block["qz"])
info["divergence_q"] = previous_block["divergence_q"]
else:
info.update({key: _RENDER_DEFAULTS[key] for key in _RENDER_DEFAULTS
if key != "coarse"})
info["coarse"] = tuple(_RENDER_DEFAULTS["coarse"])
info["size_q"] = tuple(_RENDER_DEFAULTS["size_q"])
info["qx"], info["qy"], info["qz"] = _coarse_to_q15(info["coarse"])
if reader.read(1): # b_additional_table_data_exists if reader.read(1): # b_additional_table_data_exists
info["additional_table_bytes"] = reader.read(4) + 1 info["additional_table_bytes"] = reader.read(4) + 1
reader.skip(info["additional_table_bytes"] * 8) reader.skip(info["additional_table_bytes"] * 8)
@@ -311,7 +524,7 @@ def _parse_object_info_block(reader, object_index, in_bed_or_isf, raw_payload, r
def _parse_object_element(reader, element, raw_payload, retain_spatial=False): def _parse_object_element(reader, element, raw_payload, retain_spatial=False):
"""5.5.5 ``object_element()``:时间信息 + 每个对象一条 object_info_block。""" """5.5.5 ``object_element()``:时间头 + 对象外/块内顺序的完整块状态。"""
object_count = element["object_count"] object_count = element["object_count"]
try: try:
sample_offset_code = reader.read(2) sample_offset_code = reader.read(2)
@@ -348,11 +561,20 @@ def _parse_object_element(reader, element, raw_payload, retain_spatial=False):
reserved_data_not_present = bool(reader.read(1)) reserved_data_not_present = bool(reader.read(1))
if not reserved_data_not_present: if not reserved_data_not_present:
reader.read(5) reader.read(5)
objects = [ kinds = element["object_kinds"]
_parse_object_info_block( object_blocks = []
reader, index, index < element["bed_isf_objects"], raw_payload, retain_spatial) for object_index in range(object_count):
for index in range(object_count) kind = kinds[object_index]
] states = []
previous = None
for block_index in range(block_count):
info = parse_object_info_block(
reader, object_index, block_index, kind, previous,
raw_payload, retain_spatial, object_blocks)
info["previous"] = previous
states.append(info)
previous = info
object_blocks.append(states)
except UnsupportedVariantError: except UnsupportedVariantError:
raise raise
except ValueError as exc: except ValueError as exc:
@@ -367,12 +589,216 @@ def _parse_object_element(reader, element, raw_payload, retain_spatial=False):
"sample_offset": sample_offset, "sample_offset": sample_offset,
"blocks": blocks, "blocks": blocks,
"reserved_data_not_present": reserved_data_not_present, "reserved_data_not_present": reserved_data_not_present,
"objects": objects, "objects": object_blocks,
} }
def _parse_elements(reader, alternate_present, object_count, program, raw_payload, retain_spatial=False): def _parse_extended_element(reader, element, object_element, raw_payload):
"""5.5.4 ``oa_element_md()`` 目录;对象 element 按结构解析,其余按声明长度跳过。""" """扩展对象元素:divergence 与高精度位置补充。"""
blocks = object_element["blocks"]
block_count = len(blocks)
objects = object_element["objects"]
object_count = len(objects)
if object_element is None:
raise UnsupportedVariantError(
"oamd", "extended_without_object_element",
"OAMD 扩展对象元素在默认主对象元素之前出现",
details={"element": _element_details([element])[0],
"payload": bytes_descriptor(raw_payload)})
try:
divergence_present = bool(reader.read(1))
if divergence_present:
table = DIVERGENCE_TABLE
quick = DIVERGENCE_QUICK_INDEX
for object_index in range(object_count):
for block_index in range(block_count):
state = objects[object_index][block_index]
if state["not_active"]:
state["divergence_q"] = 0
continue
if state["kind"] != "dynamic":
continue
present = bool(reader.read(1))
if not present:
state["divergence_q"] = 0
continue
mode = reader.read(2)
if mode == 0:
state["divergence_q"] = int(table[quick[reader.read(2)]])
elif mode == 1:
state["divergence_q"] = (
0 if block_index == 0
else objects[object_index][block_index - 1]["divergence_q"])
elif mode == 2:
state["divergence_q"] = int(table[reader.read(6)])
else:
reader.read(6)
state["divergence_q"] = 0
refinement_present = bool(reader.read(1))
if refinement_present:
for object_index in range(object_count):
for block_index in range(block_count):
state = objects[object_index][block_index]
if state["not_active"] or state["kind"] != "dynamic":
continue
present = bool(reader.read(1))
if not present:
continue
mask = reader.read(3)
if not mask:
continue
deltas = [0, 0, 0]
for axis in range(3):
if mask & (1 << axis):
deltas[axis] = _REFINE_DELTA[reader.read(2)]
qx, qy, qz = state["qx"], state["qy"], state["qz"]
def _sign(value):
return -1 if value < 0 else 1
def _coarse_xy(value):
return 31 + _sign(value - 16384) * (
(62 * abs(value - 16384) + 16384) >> 15)
def _coarse_z(value):
value = max(value, -32767)
return _sign(value) * ((15 * abs(value) + 16384) >> 15)
ix = _coarse_xy(qx)
iy = _coarse_xy(qy)
iz = _coarse_z(qz)
sign = _sign(iz)
qx = _refine_xy(5 * ix + 2 + deltas[0])
qy = _refine_xy(5 * iy + 2 + deltas[1])
qz = sign * _refine_z(sign * (5 * iz + deltas[2]) + 2)
state["qx"], state["qy"], state["qz"] = qx, qy, qz
except ValueError as exc:
raise _syntax_error(
"extended_element_syntax",
"OAMD 扩展对象元素字段越界或不完整",
raw_payload, exc,
element=_element_details([element])[0],
object_count=object_count,
) from exc
return {"divergence_present": divergence_present,
"refinement_present": refinement_present}
def _parse_trim_element(reader, object_count, raw_payload):
"""``trim_element``:全局 trim/warp 配置,每个 payload 出现一次。
该 element 描述整段节目的渲染校准意图,不随 block 变化,因此解析结果应当
在 payload 之间保持,而不是逐帧重算。
"""
trim = {
"warp_mode": reader.read(2),
"reserved": reader.read(2),
"global_trim_mode": reader.read(2),
"configs": [None] * 9,
"disable_per_obj": None,
}
# global_trim_mode: 0 自动、1 禁用、2 按配置显式指定、3 保留。
# 只有 2 才携带下面九个配置的显式数据。
if trim["global_trim_mode"] == 2:
for config in range(9):
if reader.read(1): # b_default_trim
trim["configs"][config] = {"default": True}
continue
if reader.read(1): # b_disable_trim
trim["configs"][config] = {"disabled": True}
continue
present = reader.read(5)
entry = {"present": present}
if present & 0x01:
entry["centre"] = TRIM_LUT[reader.read(4)]
if present & 0x02:
entry["surround"] = TRIM_LUT[reader.read(4)]
if present & 0x04:
entry["height"] = TRIM_LUT[reader.read(4)]
if present & 0x08:
sign = 1.0 if reader.read(1) else -1.0
entry["bal3d_y_tb"] = (
(reader.read(4) + 1) / BALANCE_STEP * sign)
if present & 0x10:
sign = 1.0 if reader.read(1) else -1.0
entry["bal3d_y_lis"] = (
(reader.read(4) + 1) / BALANCE_STEP * sign)
trim["configs"][config] = entry
if reader.read(1): # b_disable_trim_per_obj
trim["disable_per_obj"] = [bool(reader.read(1))
for _ in range(object_count)]
return trim
def parse_trim(bits_one):
"""独立解析一个 OAMD payload 的 trim element。
返回 ``None`` 表示该 payload 不含 trim element。异常统一转成
``UnsupportedVariantError``,与其它解析入口一致。
"""
bits, raw_payload = _payload_bits(bits_one)
try:
version = bits[0] << 1 | bits[1]
except IndexError:
raise UnsupportedVariantError(
"oamd", "header_truncated", "OAMD payload 不足以容纳 header",
details={"payload_bits": len(bits),
"payload": bytes_descriptor(raw_payload)}) from None
reader = _BitReader(bits, 2)
try:
if version == 3:
version += reader.read(3)
object_count_bits = reader.read(5)
if object_count_bits == 0x1F:
object_count_bits += reader.read(7)
object_count = object_count_bits + 1
program = _parse_program_assignment(reader, raw_payload)
except UnsupportedVariantError:
raise
except ValueError as exc:
raise _syntax_error(
"header_truncated", "OAMD header 字段越界或不完整",
raw_payload, exc, payload_bits=len(bits)) from exc
alternate_present = bool(reader.read(1))
try:
element_count = reader.read(4)
except ValueError as exc:
raise _syntax_error(
"element_count_truncated", "OAMD element 数量字段不完整",
raw_payload, exc) from exc
for ordinal in range(element_count):
header_start = reader.position
try:
element_id = reader.read(4)
size_bytes = _variable_bits_max(reader, 4, 4) + 1
except ValueError as exc:
raise _syntax_error(
"element_header_truncated", "OAMD element header 不完整",
raw_payload, exc,
element_ordinal=ordinal, header_start_bit=header_start) from exc
region_start = reader.position
region_end = region_start + size_bytes * 8
if region_end > reader.limit:
raise UnsupportedVariantError(
"oamd", "element_bounds",
"OAMD element 声明长度超过 payload 边界",
details={"element_ordinal": ordinal, "element_id": element_id,
"size_bytes": size_bytes,
"payload": bytes_descriptor(raw_payload)})
if alternate_present:
reader.read(4)
reader.read(1)
if element_id == TRIM_ELEMENT_ID:
trim = _parse_trim_element(reader, object_count, raw_payload)
trim["object_count"] = object_count
return trim
reader.position = region_end
return None
def _parse_elements(reader, alternate_present, object_count, program,
raw_payload, retain_spatial=False):
"""5.5.4 ``oa_element_md()`` 目录:先选替代变体,再解析内容。"""
try: try:
element_count = reader.read(4) element_count = reader.read(4)
if element_count == 0xF: if element_count == 0xF:
@@ -388,8 +814,18 @@ def _parse_elements(reader, alternate_present, object_count, program, raw_payloa
details={"payload": bytes_descriptor(raw_payload)}) details={"payload": bytes_descriptor(raw_payload)})
bed_isf_objects = program["num_bed_objects"] + program["num_isf_objects"] bed_isf_objects = program["num_bed_objects"] + program["num_isf_objects"]
kinds = []
for object_index in range(object_count):
if object_index < program["num_bed_objects"]:
kinds.append("bed")
elif object_index < bed_isf_objects:
kinds.append("isf")
else:
kinds.append("dynamic")
elements = [] elements = []
object_element = None object_element = None
extended = None
seen = set()
for ordinal in range(element_count): for ordinal in range(element_count):
header_start = reader.position header_start = reader.position
try: try:
@@ -415,16 +851,8 @@ def _parse_elements(reader, alternate_present, object_count, program, raw_payloa
"payload_bits": reader.limit, "payload_bits": reader.limit,
"payload": bytes_descriptor(raw_payload), "payload": bytes_descriptor(raw_payload),
}) })
try: alternate_data_id = reader.read(4) if alternate_present else 0
alternate_data_id = (reader.read(4) discard_unknown = bool(reader.read(1))
if alternate_present else None)
discard_unknown = bool(reader.read(1))
except ValueError as exc:
raise _syntax_error(
"element_control_bounds", "OAMD element 太短,无法容纳控制字段",
raw_payload, exc,
element_ordinal=ordinal, element_id=element_id,
size_bytes=size_bytes) from exc
element = { element = {
"ordinal": ordinal, "ordinal": ordinal,
"element_id": element_id, "element_id": element_id,
@@ -438,25 +866,38 @@ def _parse_elements(reader, alternate_present, object_count, program, raw_payloa
"object_count": object_count, "object_count": object_count,
"bed_isf_objects": bed_isf_objects, "bed_isf_objects": bed_isf_objects,
} }
if element_id == OBJECT_ELEMENT_ID: selected = (element_id in (OBJECT_ELEMENT_ID, TRIM_ELEMENT_ID, 5)
if object_element is not None: and alternate_data_id == 0 and element_id not in seen)
raise UnsupportedVariantError( if not selected:
"oamd", "multiple_object_elements", element["parsed_end_bit"] = region_end
"OAMD 含多个 object element", reader.position = region_end
details={ elif element_id == OBJECT_ELEMENT_ID:
"elements": _element_details(elements + [element]), element["object_kinds"] = kinds
"payload": bytes_descriptor(raw_payload),
})
object_element = _parse_object_element( object_element = _parse_object_element(
reader, element, raw_payload, retain_spatial) reader, element, raw_payload, retain_spatial)
element["parsed_end_bit"] = reader.position element["parsed_end_bit"] = reader.position
element["object_element"] = object_element element["object_element"] = object_element
# 个别编码器声明的 oa_element_size 比实际内容短;以结构解析结果为准。 seen.add(element_id)
reader.position = max(region_end, reader.position)
elif element_id == 5:
if object_element is None:
raise UnsupportedVariantError(
"oamd", "extended_before_object_element",
"OAMD 扩展对象元素出现在默认主对象元素之前",
details={"element": _element_details(elements + [element])[0],
"payload": bytes_descriptor(raw_payload)})
extended = _parse_extended_element(reader, element,
object_element, raw_payload)
element["parsed_end_bit"] = reader.position
element["extended_element"] = extended
seen.add(element_id)
reader.position = max(region_end, reader.position) reader.position = max(region_end, reader.position)
else: else:
# trim / extended / 未知 element:按声明长度整体跳过(5.5.4)。 element["trim_element"] = _parse_trim_element(
element["parsed_end_bit"] = region_end reader, object_count, raw_payload)
reader.position = region_end element["parsed_end_bit"] = reader.position
seen.add(element_id)
reader.position = max(region_end, reader.position)
elements.append(element) elements.append(element)
if object_element is None: if object_element is None:
@@ -481,6 +922,16 @@ def _parse_elements(reader, alternate_present, object_count, program, raw_payloa
return object_element, elements return object_element, elements
def _element_kinds(program, object_count):
kinds = []
bed_end = program["num_bed_objects"]
isf_end = bed_end + program["num_isf_objects"]
for index in range(object_count):
kinds.append("bed" if index < bed_end
else ("isf" if index < isf_end else "dynamic"))
return kinds
def program_bed_summary(bits_one): def program_bed_summary(bits_one):
"""只走 OAMD 头与 program_assignment(),返回 bed 声明与内容类型。 """只走 OAMD 头与 program_assignment(),返回 bed 声明与内容类型。
@@ -521,6 +972,11 @@ def program_bed_summary(bits_one):
"num_bed_objects": program["num_bed_objects"], "num_bed_objects": program["num_bed_objects"],
"num_isf_objects": program["num_isf_objects"], "num_isf_objects": program["num_isf_objects"],
"num_dynamic_objects": program["num_dynamic_objects"], "num_dynamic_objects": program["num_dynamic_objects"],
"isf_idx": program["isf_idx"],
"dynamic_object_only": program["dynamic_object_only"],
"lfe_present": program["lfe_present"],
"b_bed_chan_distribute": program["b_bed_chan_distribute"],
"bed_mask": program["bed_mask"],
} }
@@ -533,8 +989,8 @@ def program_bed_channels(bits_one):
return program_bed_summary(bits_one)["channels"] return program_bed_summary(bits_one)["channels"]
def frame_update(bits_one, *, retain_spatial=False): def frame_state(bits_one, *, retain_spatial=False):
"""单帧 OAMD → 位置字段增量及其 sample offset/ramp duration。""" """完整展开一帧 OAMD:每个对象每一块的属性、节目分区与扩展对象元素更新。"""
bits, raw_payload = _payload_bits(bits_one) bits, raw_payload = _payload_bits(bits_one)
try: try:
version = bits[0] << 1 | bits[1] version = bits[0] << 1 | bits[1]
@@ -555,14 +1011,14 @@ def frame_update(bits_one, *, retain_spatial=False):
program = _parse_program_assignment(reader, raw_payload) program = _parse_program_assignment(reader, raw_payload)
alternate_present = bool(reader.read(1)) alternate_present = bool(reader.read(1))
object_element, elements = _parse_elements( object_element, elements = _parse_elements(
reader, alternate_present, object_count, program, raw_payload, retain_spatial) reader, alternate_present, object_count, program, raw_payload,
retain_spatial)
except UnsupportedVariantError: except UnsupportedVariantError:
raise raise
except ValueError as exc: except ValueError as exc:
raise _syntax_error( raise _syntax_error(
"header_truncated", "OAMD header 字段越界或不完整", "header_truncated", "OAMD header 字段越界或不完整",
raw_payload, exc, payload_bits=len(bits)) from exc raw_payload, exc, payload_bits=len(bits)) from exc
if version != 0: if version != 0:
raise UnsupportedVariantError( raise UnsupportedVariantError(
"oamd", "oamd_version", "oamd", "oamd_version",
@@ -571,46 +1027,27 @@ def frame_update(bits_one, *, retain_spatial=False):
"version": version, "version": version,
"payload": bytes_descriptor(raw_payload), "payload": bytes_descriptor(raw_payload),
}) })
if object_count > MAX_SLOTS:
raise UnsupportedVariantError(
"oamd", "object_count",
"OAMD 对象数超过当前 16 槽对象模型",
details={
"object_count": object_count,
"max_slots": MAX_SLOTS,
"payload": bytes_descriptor(raw_payload),
})
blocks = object_element["blocks"] blocks = object_element["blocks"]
if len(blocks) != 1: objects = object_element["objects"]
raise UnsupportedVariantError( # 对外契约:16 槽位置视图(block 0),保留旧 frame_update 的 values 形状。
"oamd", "multiple_position_blocks", values = {(slot, field): None for slot in range(MAX_SLOTS)
"OAMD 一帧含多个对象位置更新块,单次 frame_update 无法表达", for field in ("q1", "q2", "q3")}
details={ for object_index, states in enumerate(objects):
"block_count": len(blocks), if object_index >= MAX_SLOTS:
"blocks": blocks, break
"payload": bytes_descriptor(raw_payload), state = states[0]
"repair_hint": "按 ObjectInfoBlock 顺序逐块解析坐标,再生成分段 ADM ramp", if state["not_active"] or state.get("position") is None and \
}) state["render_status"] == 0 and state["kind"] != "dynamic":
# bed/ISF 与未激活对象没有位置字段:保持上一帧位置。
# 对外契约:始终给出 16 个槽;本帧没有覆盖到的槽保持上一帧位置。
out = {(slot, field): None for slot in range(MAX_SLOTS)
for field in ("q1", "q2", "q3")}
for info in object_element["objects"]:
slot = info["object"]
position = info.get("position")
if position is None:
# bed/ISF 对象与未激活对象没有位置字段:保持上一帧位置。
out[(slot, "q1")] = None
out[(slot, "q2")] = None
out[(slot, "q3")] = None
continue continue
x, y, z = position values[(object_index, "q1")] = state["qx"]
out[(slot, "q1")] = q_of(x, N_Q12) if 0 <= x <= N_Q12 else None values[(object_index, "q2")] = state["qy"]
out[(slot, "q2")] = q_of(y, N_Q12) if 0 <= y <= N_Q12 else None qz = state["qz"]
# Preserve legacy nonnegative height unless signed spatial output is requested. if not state["enable_elevation"]:
out[(slot, "q3")] = q_of(z, N_Q3) if (0 <= z <= N_Q3 or retain_spatial and -N_Q3 <= z < 0) else ( # 描述符构建在 enable_elevation=0 时令 z=0。
0 if z < 0 else None) qz = 0
values[(object_index, "q3")] = (
qz if qz >= 0 or retain_spatial else 0)
size_mismatch = [ size_mismatch = [
{ {
"element_ordinal": element["ordinal"], "element_ordinal": element["ordinal"],
@@ -622,11 +1059,18 @@ def frame_update(bits_one, *, retain_spatial=False):
for element in elements for element in elements
if element["parsed_end_bit"] > element["body_end_bit"] if element["parsed_end_bit"] > element["body_end_bit"]
] ]
trim = next((element["trim_element"] for element in elements
if "trim_element" in element), None)
return { return {
"values": out, "values": values,
"block_offset_samples": blocks[0]["block_offset_samples"], "block_offset_samples": blocks[0]["block_offset_samples"],
"ramp_duration_samples": blocks[0]["ramp_duration_samples"], "ramp_duration_samples": blocks[0]["ramp_duration_samples"],
"blocks": blocks,
"sample_offset": object_element["sample_offset"],
"object_count": object_count, "object_count": object_count,
"kinds": [states[0]["kind"] for states in objects],
"objects": [dict(states[0]) for states in objects],
"object_blocks": objects,
"program": { "program": {
"dynamic_object_only": program["dynamic_object_only"], "dynamic_object_only": program["dynamic_object_only"],
"lfe_present": program["lfe_present"], "lfe_present": program["lfe_present"],
@@ -636,20 +1080,48 @@ def frame_update(bits_one, *, retain_spatial=False):
"num_dynamic_objects": program["num_dynamic_objects"], "num_dynamic_objects": program["num_dynamic_objects"],
"bed_channels": [list(assignment["channels"]) "bed_channels": [list(assignment["channels"])
for assignment in program["bed_assignments"]], for assignment in program["bed_assignments"]],
"bed_mask": program["bed_mask"],
"b_bed_chan_distribute": program["b_bed_chan_distribute"],
"isf_idx": program["isf_idx"],
}, },
"trim": trim,
"elements": _element_details(elements), "elements": _element_details(elements),
"objects": object_element["objects"],
"size_mismatch": size_mismatch, "size_mismatch": size_mismatch,
} }
def frame_update(bits_one, *, retain_spatial=False):
"""单帧 OAMD → 位置字段增量及其 sample offset/ramp duration。"""
state = frame_state(bits_one, retain_spatial=retain_spatial)
if state["object_count"] > MAX_SLOTS:
raise UnsupportedVariantError(
"oamd", "object_count",
"OAMD 对象数超过当前 16 槽对象模型",
details={
"object_count": state["object_count"],
"max_slots": MAX_SLOTS,
})
if len(state["blocks"]) != 1:
# 旧接口(ADM/双耳/sofa 轨迹)只表达单块位置;多块由 frame_state 提供给
# 扬声器渲染路径。这里保持与旧实现一致的显式拒绝,避免静默只取首块。
raise UnsupportedVariantError(
"oamd", "multiple_position_blocks",
"OAMD 一帧含多个对象位置更新块,单次 frame_update 无法表达",
details={
"block_count": len(state["blocks"]),
"blocks": state["blocks"],
"repair_hint": "扬声器渲染路径改用 frame_state()",
})
return state
def frame_update_values(bits_one): def frame_update_values(bits_one):
"""兼容接口:只返回 ``{(slot, field): q|None}``。""" """兼容接口:只返回 ``{(slot, field): q|None}``。"""
return frame_update(bits_one)["values"] return frame_update(bits_one)["values"]
class JocFieldState: class JocFieldState:
"""未更新/非法窗保持旧值;slot0 q1/q2 的 DLL 初值为中心 16384。""" """未更新/非法窗保持旧值;slot0 q1/q2 初始为中心 16384。"""
def __init__(self): def __init__(self):
self.q = {(obj, field): 0 for obj in range(16) self.q = {(obj, field): 0 for obj in range(16)
+699
View File
@@ -0,0 +1,699 @@
"""Float32 equations for the metadata application path.
The routines implement the fixed float32 equations for the size tables, ISF
basis assembly, bed channel routing and the position-dependent compensation.
The size tables are generated from the region geometry with grouped equal-power
panning, an S448 weight and a power sum; every operation is evaluated in
float32 in a fixed order so the coefficients are reproducible.
The caller passes the parameter mappings from :mod:`oamd_params`; this module
does not read files itself.
"""
from __future__ import annotations
import math
from functools import lru_cache
import numpy as np
from bed_layout import BED_REFERENCE_POSITIONS_Q15, bed_output_description
from speaker_layouts import get_speaker_layout
F = np.float32
TINY = F(2 ** -126)
def exp2_approx(value):
"""Specified ``2**x`` approximation used by the compensation kernels."""
x = F(value)
n = math.floor(float(x))
f = F(x - F(n))
poly = F(F(F(F(f * F(0.6862793)) + F(1)) + F(F(f * f) * F(0.25482178)))
+ F(F(f * F(f * f)) * F(0.058898926)))
return F(math.ldexp(float(poly), n))
def pow_approx(x, power):
"""Specified ``x**power`` approximation; ``power`` is a float32."""
x = F(x)
if abs(x) < TINY:
return F(0)
fraction, exponent = math.frexp(float(x))
fraction = F(fraction)
logarithm = F(F(F(F(fraction * F(4.0793457)) - F(2.6931152))
- F(F(fraction * fraction) * F(1.3862305))) + F(exponent))
return exp2_approx(F(logarithm * F(power)))
def norm(vector):
"""Stable float32 normalization with the tiny-norm floor."""
vector = np.asarray(vector, dtype=np.float32)
total = F(0)
for value in vector:
total = F(total + F(value * value))
if abs(total) < TINY:
total = F(0)
denominator = max(F(math.sqrt(float(total))), F(1e-5))
result = (vector / denominator).astype(np.float32)
result[np.abs(result) < TINY] = 0
return result
def boundary(axis, edge, position, t):
"""One boundary weight of the size kernel."""
edge, position, t = F(edge), F(position), F(t)
delta = F(F(edge - position) * F(0.25))
if t < F(0.000025) or abs(delta) > t:
return F(0)
ratio = F(float(delta) / float(t))
ratio2 = F(ratio * ratio)
weight = exp2_approx(F(F(-F(ratio2 * ratio2) * F(0.6328125))
* F(26.575424)))
_, exponent = math.frexp(float(weight))
if exponent < -31 or exponent > 1:
weight = F(0)
if axis == 2:
weight = F(F(math.cos(float(F(F(edge * F(0.34147748)) * F(4)))))
* weight)
return weight
def _region_coordinates(region):
"""Normalize the region's Q15 coordinates to the 0..1 room grid."""
return (np.asarray(region.coordinates_q15, dtype=np.float32)
/ F(32768))
def _group_pan_x(coordinates, groups, position, output):
"""Region X panning: equal-power between x-adjacent speakers."""
position = F(position)
for group in groups:
first = group[0]
last = group[-1]
x_first = coordinates[first][0]
x_last = coordinates[last][0]
if position < x_first:
output[first] = F(1)
continue
if position > x_last:
output[last] = F(1)
continue
for left, right in zip(group, group[1:]):
x_left = coordinates[left][0]
x_right = coordinates[right][0]
if not (x_left < x_right and x_left <= position <= x_right):
continue
fraction = F(F(position - x_left) / F(x_right - x_left))
if fraction <= F(0.5):
angle = F(F(F(fraction * F(0.78539819)) * F(0.5)) * F(4))
output[left] = F(np.cos(np.float32(angle)))
output[right] = F(np.sin(np.float32(angle)))
else:
angle = F(F(F(F(1) - fraction) * F(0.78539819))
* F(0.5) * F(4))
output[left] = F(np.sin(np.float32(angle)))
output[right] = F(np.cos(np.float32(angle)))
break
def _group_pan_y(coordinates, groups, position, output):
"""Region Y panning: equal-power between y-adjacent speaker rows."""
if not groups:
return
position = F(position)
rows = [coordinates[group[0]][1] for group in groups]
if position <= rows[0]:
for index in groups[0]:
output[index] = F(1)
return
if position > rows[-1]:
for index in groups[-1]:
output[index] = F(1)
return
for row in range(len(groups) - 1):
y_low = rows[row]
y_high = rows[row + 1]
if not (y_low <= position <= y_high and y_low < y_high):
continue
fraction = F(F(position - y_low) / F(y_high - y_low))
if fraction <= F(0.5):
angle = F(F(F(fraction * F(0.78539819)) * F(0.5)) * F(4))
low_gain = F(np.cos(np.float32(angle)))
high_gain = F(np.sin(np.float32(angle)))
else:
angle = F(F(F(F(1) - fraction) * F(0.78539819))
* F(0.5) * F(4))
low_gain = F(np.sin(np.float32(angle)))
high_gain = F(np.cos(np.float32(angle)))
for index in groups[row]:
output[index] = low_gain
for index in groups[row + 1]:
output[index] = high_gain
return
def _layer_pan_z(coordinates, position):
"""Region Z panning: floor/height equal-power split."""
position = F(position)
if position <= F(0):
floor_gain = F(1)
height_gain = F(0)
elif position < F(0.99996948):
angle = F(F(position * F(0.39269909)) * F(4))
floor_gain = F(np.cos(np.float32(angle)))
height_gain = F(np.sin(np.float32(angle)))
else:
floor_gain = F(0)
height_gain = F(1)
output = np.zeros(len(coordinates), dtype=np.float32)
for index, coordinate in enumerate(coordinates):
output[index] = (height_gain if coordinate[2] >= F(0.5)
else floor_gain)
return output
def _pan_x_all(coordinates, floor_groups, height_groups, position, mode):
output = np.zeros(len(coordinates), dtype=np.float32)
_group_pan_x(coordinates, floor_groups, position, output)
if mode == 3 and height_groups:
extra = np.zeros(len(coordinates), dtype=np.float32)
_group_pan_x(coordinates, height_groups, position, extra)
output = F(output + extra)
return output
def _pan_y_all(coordinates, floor_groups, height_groups, position, mode):
output = np.zeros(len(coordinates), dtype=np.float32)
_group_pan_y(coordinates, floor_groups, position, output)
if mode == 3 and height_groups:
extra = np.zeros(len(coordinates), dtype=np.float32)
_group_pan_y(coordinates, height_groups, position, extra)
output = F(output + extra)
return output
def _boundary_rows(coordinates, floor_groups, height_groups, mode):
x0 = _pan_x_all(coordinates, floor_groups, height_groups, F(0), mode)
x1 = _pan_x_all(coordinates, floor_groups, height_groups, F(1), mode)
y0 = _pan_y_all(coordinates, floor_groups, height_groups, F(0), mode)
y1 = _pan_y_all(coordinates, floor_groups, height_groups, F(1), mode)
if mode == 3:
z1 = _layer_pan_z(coordinates, F(1))
else:
z1 = np.zeros(len(coordinates), dtype=np.float32)
return [x0, x1, y0, y1, z1]
def _size_weight(position, u, t, axis):
"""Per-virtual-point S448 weight used by the generated size tables."""
t = F(t)
offset = F(F(F(u - position)) * F(0.25))
if t < F(0.000025) or abs(float(offset)) > float(t):
return F(0)
ratio = F(offset / t)
squared = F(ratio * ratio)
power = F(F(-F(squared * squared)) * F(0.6328125))
weight = exp2_approx(F(power * F(26.575424)))
if axis == 2:
angle = F(F(F(u * F(0.34147748))) * F(4))
weight = F(F(np.cos(np.float32(angle))) * weight)
return weight
def _axis_table(pan_vectors, position_nodes, speaker_count, offset_start,
axis):
table = np.zeros((position_nodes, 20, speaker_count), dtype=np.float32)
for position_index in range(position_nodes):
position = F(F(position_index) / F(position_nodes - 1))
for size_index in range(20):
t = F(F(size_index) * F(0.036842104))
m = (F(6) if t <= F(0.125)
else F(F(6.869565) - F(t * F(6.956522))))
for u, pan in enumerate(pan_vectors):
weight = _size_weight(position, F(F(u) / F(len(pan_vectors) - 1)),
t, axis)
if weight == F(0):
continue
for speaker in range(speaker_count):
value = F(pan[offset_start + speaker] * weight)
table[position_index, size_index, speaker] = F(
table[position_index, size_index, speaker]
+ pow_approx(value, m))
return table
def _axis2_table(pan_vectors, speaker_index):
table = np.zeros((4, 20), dtype=np.float32)
if speaker_index >= len(pan_vectors[0]):
return table
for position_index in range(4):
position = F(F(position_index) / F(3))
for size_index in range(20):
t = F(F(size_index) * F(0.036842104))
m = (F(6) if t <= F(0.125)
else F(F(6.869565) - F(t * F(6.956522))))
total = F(0)
for u, pan in enumerate(pan_vectors):
weight = _size_weight(position, F(F(u) / F(7)), t, 2)
if weight == F(0):
continue
total = F(total + pow_approx(
F(pan[speaker_index] * weight), m))
table[position_index, size_index] = total
return table
@lru_cache(maxsize=256)
def _region_size_tables(region):
"""Generate the region size tables from the region geometry."""
coordinates = _region_coordinates(region)
floor_groups = tuple(region.axis0_groups)
height_groups = tuple(region.axis1_groups) if region.mode == 3 else ()
floor_count = sum(len(group) for group in floor_groups)
height_count = sum(len(group) for group in height_groups)
mode = region.mode
x_vectors = [_pan_x_all(coordinates, floor_groups, height_groups,
F(F(index) / F(19)), mode) for index in range(20)]
y_vectors = [_pan_y_all(coordinates, floor_groups, height_groups,
F(F(index) / F(19)), mode) for index in range(20)]
z_vectors = ([_layer_pan_z(coordinates, F(F(index) / F(7)))
for index in range(8)] if mode == 3
else [np.zeros(len(coordinates), dtype=np.float32)
for _ in range(8)])
x_floor = _axis_table(x_vectors, 35, floor_count, 0, 0)
x_height = _axis_table(x_vectors, 35, height_count, floor_count, 0)
y_floor = _axis_table(y_vectors, 35, floor_count, 0, 1)
y_height = _axis_table(y_vectors, 35, height_count, floor_count, 1)
if mode == 3:
z_floor = _axis2_table(z_vectors, 0)
z_height = _axis2_table(z_vectors, floor_count)
else:
z_floor = np.zeros(0, dtype=np.float32)
z_height = np.zeros(0, dtype=np.float32)
entry = {
"floor_count": floor_count,
"height_count": height_count,
"mode": mode,
"floor_axis_group_count": len(floor_groups),
"height_axis_group_count": len(height_groups),
"boundary_3d": _boundary_rows(coordinates, floor_groups,
height_groups, mode),
"boundary_2d": _boundary_rows(coordinates, floor_groups, (), 2),
"axis_tables": {
"0": {"position_nodes": 35, "size_nodes": 20,
"floor_stride": floor_count,
"height_stride": height_count,
"floor": x_floor.reshape(-1).copy(),
"height": x_height.reshape(-1).copy()},
"1": {"position_nodes": 35, "size_nodes": 20,
"floor_stride": floor_count,
"height_stride": height_count,
"floor": y_floor.reshape(-1).copy(),
"height": y_height.reshape(-1).copy()},
"2": {"position_nodes": 4, "size_nodes": 20,
"floor_stride": 1, "height_stride": 1,
"floor": z_floor.reshape(-1).copy(),
"height": z_height.reshape(-1).copy()},
},
}
return entry
def _bracket(position, axis):
position = F(position)
step = F(F(1) / F(3 if axis == 2 else 34))
last = 3 if axis == 2 else 34
low = math.floor(float(F(F(last) * position)))
high = min(low + 1, last)
if low > last:
low = high = last
alpha = (F(0) if low == high
else max(F(0), F(F(position - F(F(low) * step)) / step)))
return low, high, alpha
def _size_bracket(t):
t = F(t)
step = F(0.036842104)
low = math.floor(float(F(F(F(t * F(0.71428573)) * F(2)) * F(19))))
if low > 19:
return 19, 19, F(0)
high = min(low + 1, 19)
beta = (F(0) if low == high
else max(F(0), F(float(F(t - F(F(low) * step))) / float(step))))
return low, high, beta
def _axis_weights(entry, axis, position, t, height):
low, high, alpha = _bracket(position, axis)
low_t, high_t, beta = _size_bracket(t)
weights = np.asarray([
F(1) - max(beta, alpha),
F(1) - max(beta, F(1) - alpha),
F(1) - max(F(1) - beta, alpha),
F(1) - max(F(1) - beta, F(1) - alpha)], dtype=np.float32)
total = F(F(F(weights[1] + weights[0]) + weights[2]) + weights[3])
weights = np.asarray([F(float(value) / float(total)) for value in weights],
dtype=np.float32)
table = entry["axis_tables"][str(axis)]
result = []
for group, count in (("floor", entry["floor_count"]),
("height", entry["height_count"] if height else 0)):
if not count:
continue
stride = table[group + "_stride"]
values = np.asarray(table[group], dtype=np.float32).reshape(
table["position_nodes"], 20, stride)
value = F(F(F(weights[0] * values[low, low_t]
+ weights[1] * values[high, low_t])
+ weights[2] * values[low, high_t])
+ weights[3] * values[high, high_t])
result.extend([value[0]] * count if axis == 2 else value[:count])
return np.asarray(result, dtype=np.float32)
def size_kernel(region, position, t, height=True):
"""Size kernel response in region-speaker order."""
entry = _region_size_tables(region)
position = np.asarray(position, dtype=np.float32)
t = F(t)
floor_count = entry["floor_count"]
height_count = entry["height_count"] if height else 0
total = floor_count + height_count
if not total:
return np.zeros(0, dtype=np.float32)
dim = (entry["mode"] if height
else (1 if entry["floor_axis_group_count"] == 1 else 2))
m = (F(6) if t <= F(0.125)
else F(F(6.869565) - F(t * F(6.956522))))
inverse_m = F(F(1) / m)
axes = [_axis_weights(entry, axis, position[axis], t, height)
for axis in range(dim)]
key = "boundary_3d" if dim == 3 else "boundary_2d"
boundary_table = np.asarray(entry[key], dtype=np.float32)[
[4, 0, 1, 2, 3], :total].copy()
for row, (axis, edge) in enumerate([(2, 1), (0, 0), (0, 1),
(1, 0), (1, 1)]):
boundary_table[row] *= boundary(axis, edge, position[axis], t)
inside = axes[0].copy()
if dim > 1:
inside *= axes[1]
if dim > 2:
inside *= axes[2]
inside = norm(inside)
if dim == 1:
return np.asarray([pow_approx(value, inverse_m) for value in inside],
dtype=np.float32)
powered = np.asarray([[pow_approx(value, m) for value in row]
for row in boundary_table], dtype=np.float32)
x, y = axes[0], axes[1]
if dim == 2:
edge = F(F(F(powered[1] * y + powered[2] * y)
+ powered[3] * x) + powered[4] * x)
else:
z = axes[2]
edge = F(F(F(F(F(x * y) * powered[0]
+ F(y * z) * powered[1])
+ F(y * z) * powered[2])
+ F(x * z) * powered[3])
+ F(x * z) * powered[4])
edge = norm(edge)
distances = [position[0], F(1) - position[0],
position[1], F(1) - position[1]]
if dim == 3:
distances.append(F(1) - position[2])
distance = min(distances)
if t < F(0.000025) or (F(F(0.25) * distance) > t
and F(F(0.25) * distance) > F(0.05)):
mix = F(1)
else:
mix = F(F(float(distance) / float(F(t * F(4))))
* F(F(distance * F(5)) * F(distance * F(5))))
joined = F(F(F(mix * inside) + edge) * F(0.5)) * F(0.0625)
return np.asarray([pow_approx(value, inverse_m) for value in joined],
dtype=np.float32)
_SIZE_CURVE_NODES = ((0.0, 0.0), (0.2, 0.075), (0.5, 0.25),
(0.75, 0.45), (1.0, 0.7))
def size_curve_value(size):
"""Map a scalar size in ``[0, 1]`` to ``t`` on the size curve."""
points = [(F(point[0]), F(point[1])) for point in _SIZE_CURVE_NODES]
size = F(size)
if size <= points[0][0]:
return F(0)
if size >= points[-1][0]:
return F(1)
for (size_low, t_low), (size_high, t_high) in zip(points, points[1:]):
if size <= size_high:
span = F(size_high - size_low)
fraction = F(F(size - size_low) / span) if span else F(0)
return F(t_low + F(fraction * F(t_high - t_low)))
return F(1)
def size_targets(meta, layout, region_index, position, size, height=True):
"""Return a size-kernel target vector in ``layout`` channel order."""
region = layout.regions[int(region_index)]
result = np.zeros(layout.channel_count, dtype=np.float32)
if not len(region.speaker_ids):
return result
t = size_curve_value(size)
kernel = size_kernel(region, position, t, height=height)
for value, channel in zip(kernel, region.speaker_ids):
result[channel] = value
return result
def assemble_isf(meta, layout, isf_idx):
"""Assemble one ISF basis matrix from the compressed base tables."""
config = meta["isf"]["configs"][isf_idx]
geometry = meta["isf"]["layouts"][layout]
row_count = config["counts"][4]
matrix = np.asarray(config["matrix_values"], dtype=np.float32).reshape(
-1, 73)
result = np.zeros((row_count, geometry["output_count"]), dtype=np.float32)
cursor = 0
def add_group(name, selector):
nonlocal cursor
record = next((item for item in meta["isf"]["layout_groups"][name]
if item[0] == selector), None)
if record is None:
return
for column in range(record[1], record[2]):
mappings = [config["primary_row_map"]]
if meta["isf"]["paired_columns"][column]:
mappings.append(config["secondary_row_map"])
for mapping in mappings:
speaker = geometry["output_ids"][cursor]
result[:, speaker] += matrix[np.asarray(mapping), column]
cursor += 1
floor_a, floor_b, height, additional = geometry["selectors"]
add_group("floor_a", floor_a)
cursor -= 2
add_group("floor_b", floor_b)
cursor -= 2
add_group("height", height)
if additional:
mid, upper, lower, zenith, _ = config["counts"]
result[mid:mid + upper] = 0
result[mid + upper + lower:mid + upper + lower + zenith] = 0
add_group("additional", additional)
return result
def isf_matrix(meta, layout, isf_idx):
"""Return the assembled ISF basis matrix ``(Nisf, output_count)``."""
return assemble_isf(meta, layout, isf_idx)
def isf_compensation(meta, layout, isf_idx, row, disabled=False):
"""Return the per-basis ISF compensation gain."""
if disabled:
return F(1)
config = meta["isf"]["configs"][isf_idx]
geom = meta["isf"]["layouts"][layout]
first = config["counts"][0]
value = F(geom["compensation_value"])
if row < first:
distance = F(min(row, first - row))
lower = max(F(F(first) * F(0.16666667)), F(1))
width = F(F(first) * F(0.11111111))
if distance <= lower:
fraction = F(0)
elif distance > F(lower + width):
fraction = F(1)
else:
fraction = F(F(distance - lower) / width)
value = F(fraction * value)
return exp2_approx(F(value * F(0.1660964)))
DIRECT_BED_LABEL = {1: 0, 2: 1, 3: 2, 4: 3, 5: 4, 6: 5, 7: 6, 8: 7,
9: 13, 10: 14, 11: 17, 12: 18, 13: 21, 14: 22,
15: 11, 16: 12, 17: 3}
A_WEIGHT = F(0.5011872053146362)
B_WEIGHT = F(0.7079457640647888)
C_WEIGHT = F(0.3548133969306946)
def bed_weight(label, mask, target_label):
"""Return the per-label bed redistribution weight."""
value = F(1 if target_label == DIRECT_BED_LABEL.get(label, 3) else 0)
if 9 <= label <= 14:
rows = {9: {13: 1, 15: B_WEIGHT, 17: A_WEIGHT},
10: {14: 1, 16: B_WEIGHT, 18: A_WEIGHT},
11: {13: A_WEIGHT, 15: B_WEIGHT, 17: 1,
19: B_WEIGHT, 21: A_WEIGHT},
12: {14: A_WEIGHT, 16: B_WEIGHT, 18: 1,
20: B_WEIGHT, 22: A_WEIGHT},
13: {17: A_WEIGHT, 19: B_WEIGHT, 21: 1},
14: {18: A_WEIGHT, 20: B_WEIGHT, 22: 1}}
return F(rows[label].get(target_label, value))
if mask & 0xC0:
rows = {5: {4: 1, 25: B_WEIGHT, 27: B_WEIGHT},
6: {5: 1, 26: B_WEIGHT, 28: B_WEIGHT},
7: {6: 1, 8: C_WEIGHT, 29: B_WEIGHT, 31: B_WEIGHT,
33: A_WEIGHT},
8: {7: 1, 8: C_WEIGHT, 30: B_WEIGHT, 32: B_WEIGHT,
34: A_WEIGHT}}
elif mask & 0x30:
rows = {5: {4: 1, 6: F(0.5370317697525024),
8: F(0.2884031534194946), 25: B_WEIGHT,
27: F(0.8128305077552795),
29: F(0.6606934666633606),
31: F(0.43651583790779114), 33: C_WEIGHT},
6: {5: 1, 7: F(0.5370317697525024),
8: F(0.2884031534194946), 26: B_WEIGHT,
28: F(0.8128305077552795),
30: F(0.6606934666633606),
32: F(0.43651583790779114), 34: C_WEIGHT}}
else:
return value
if label == 5 and not mask & 0x4000 and target_label == 11:
return A_WEIGHT
if label == 6 and not mask & 0x8000 and target_label == 12:
return A_WEIGHT
return F(rows.get(label, {}).get(target_label, value))
def bed_candidates(label, mask, output_labels):
"""Return the five height-candidate flags for one input bed label."""
available = [int(value in output_labels)
for value in (13, 15, 17, 19, 21)]
a, b, m, d, e = available
remaining = available.copy()
bits = (8, 10, 12) if label in (9, 11, 13) else (9, 11, 13)
front, middle, rear = [(mask >> bit) & 1 for bit in bits]
if front:
if a:
remaining[0] = 0
elif b:
remaining[1] = 0
if middle and m:
remaining[2] = 0
if rear:
if e:
remaining[4] = 0
elif d:
remaining[3] = 0
special = (not m and sum(available) == 2 and not front and not rear
and middle and (a or b) and (d or e))
if label in (9, 10):
return [a, b, remaining[2], 0, 0] if (a or b or remaining[2]) \
else [0] * 5
if label in (11, 12):
return [0] * 5 if special else [remaining[0], remaining[1], m,
remaining[3], remaining[4]]
if label in (13, 14):
return [0, 0, remaining[2], d, e] if (d or e or remaining[2]) \
else [0] * 5
return [0] * 5
def bed_distribution(layout_key, label, mask, distribute):
"""Return ``(route, weights)`` for one input bed label.
``route == -1`` means the returned weights are the target; otherwise
``route`` is the reference position used to synthesize a point source.
The slot map is derived from the speaker layout, not stored separately.
"""
layout = bed_output_description(get_speaker_layout(layout_key))
labels = layout["output_labels"]
base = np.zeros(len(labels), dtype=np.float32)
slot = layout["label_to_slot"][label - 1]
fallback = layout["direct_output_labels"][label - 1]
height = 9 <= label <= 14
if slot == -2 and (not height or not distribute):
return fallback, base
if slot == -1:
return -1, base
if slot >= 0:
base[slot] = F(1)
if not distribute:
return -1, base
if height:
candidate = bed_candidates(label, mask, labels)
if not any(candidate):
return fallback, base
side_labels = (13, 15, 17, 19, 21) if label % 2 \
else (14, 16, 18, 20, 22)
allowed = {value for value, active in zip(side_labels, candidate)
if active}
weights = np.array([bed_weight(label, mask, value)
if value in allowed else F(0)
for value in labels], dtype=np.float32)
else:
weights = np.array([bed_weight(label, mask, value)
for value in labels], dtype=np.float32)
if all(value == F(0) or value == F(1) for value in weights):
return -1, base
count = np.count_nonzero(weights)
if count == 1:
weights[weights != 0] = F(1)
elif count > 1:
weights = norm(weights)
else:
return fallback, base
return -1, weights
def bed_reference_position(layout_key, index):
"""Return the Q15 reference position used by the bed fallback path."""
return BED_REFERENCE_POSITIONS_Q15[int(index)]
def bed_compensation(layout_key, label, mode=0, bypass=False,
levels=(0, 0, 0)):
"""Return the bed path compensation gain for one input bed label."""
if label in (4, 17) or bypass or mode == 1:
return F(1)
layout = bed_output_description(get_speaker_layout(layout_key))
source = layout["direct_output_labels"][label - 1]
position = BED_REFERENCE_POSITIONS_Q15[source]
x, y, z = [F(value / 32768) for value in position]
if mode == 0:
y_term = F(1) if y >= F(0.6) else (
F(0) if y <= 0 else F(float(y) / 0.6))
z_term = F(1) if z >= 1 else (
F(0) if z <= F(0.2) else F(float(F(z - F(0.2))) / 0.8))
amount = max(F(0), min(F(y_term + z_term), F(1)))
value_db = F(amount * F(layout["automatic_attenuation_db"]))
else:
surround, height, centre = map(F, levels)
surround_factor = F(surround if y >= F(0.5)
else F(F(y * surround) * F(2)))
centre_position = max(F(abs(z) - F(0.05)), F(y - F(0.05)),
F(abs(F(x - F(0.5))) - F(0.05)))
centre_factor = F(F(1) - min(max(F(10) * centre_position, F(0)),
F(1)))
value_db = F(F(F(surround_factor * F(F(1) - z))
+ F(z * height)) + F(centre_factor * centre))
return exp2_approx(F(value_db * F(0.1660964)))
+27
View File
@@ -0,0 +1,27 @@
"""Static lookup data used by the OAMD metadata application path.
``metadata_params.json``
ISF base coefficients and layout descriptors, applied as lookup tables.
gain, size, priority, position, delta and refinement values are computed
in code; the size tables are generated from the region geometry at runtime,
and bed routing is derived from the speaker layouts.
The data file is parsed once per process.
"""
from __future__ import annotations
import json
from functools import lru_cache
from pathlib import Path
_DATA_DIR = Path(__file__).resolve().parent.parent / "data" / "oamd"
@lru_cache(maxsize=1)
def metadata_params():
"""Return the parsed ``metadata_params.json`` mapping."""
with (_DATA_DIR / "metadata_params.json").open(
"r", encoding="utf-8") as handle:
return json.load(handle)
+84 -22
View File
@@ -3,7 +3,7 @@ import math
from adm_atmos import q_to_adm_xyz from adm_atmos import q_to_adm_xyz
from bed_layout import bed_object_positions from bed_layout import bed_object_positions
from oamd_bits import JocFieldState, frame_update from oamd_bits import JocFieldState, frame_state, frame_update, parse_trim
from variant_error import UnsupportedVariantError from variant_error import UnsupportedVariantError
# 元数据更新时刻按渲染器处理块对齐,与 Dolby 的 processing_block_size 及 # 元数据更新时刻按渲染器处理块对齐,与 Dolby 的 processing_block_size 及
@@ -19,12 +19,42 @@ def align_metadata_sample(sample, block_samples=METADATA_BLOCK_SAMPLES):
return block * ((int(sample) + block // 2 - 1) // block) return block * ((int(sample) + block // 2 - 1) // block)
def _default_meta():
"""ADM 缺省渲染参数:点源、0 dB、无 divergence。"""
return {"gain": 1.0, "size": (0.0, 0.0, 0.0), "divergence": 0.0,
"snap": False}
def _apply_state(meta, state):
"""把一帧 OAMD 对象状态合并进跨帧保持的渲染参数。
语义与 ``PythonSpeakerRenderer`` 的 metadata 路径一致:动态对象 block 0
的 gain/size/snap 每帧都显式出现;divergence 只在扩展元素里出现,缺失时
渲染器按 0 处理,因此这里也直接取 0,而不是保持旧值。未激活对象由渲染器
静音,ADM 对象轨本身仍携带 PCM,必须写 gain=0。
"""
if state.get("kind") != "dynamic":
return
if state.get("not_active"):
meta["gain"] = 0.0
return
if state.get("render_status") == 1:
size = (0.0, 0.0, 0.0)
if state.get("size_mode") in (1, 2):
size = tuple(min(max(float(value) / 32767.0, 0.0), 1.0)
for value in state.get("size_q", (0, 0, 0)))
meta["gain"] = float(state.get("gain_q", 4096)) / 4096.0
meta["size"] = size
meta["snap"] = bool(state.get("snap", 0))
meta["divergence"] = float(state.get("divergence_q", 0)) / 32767.0
def _lerp_xyz(start, target, amount): def _lerp_xyz(start, target, amount):
return tuple(a + (b - a) * amount for a, b in zip(start, target)) return tuple(a + (b - a) * amount for a, b in zip(start, target))
def _append_point(points, sample, xyz, interpolation_samples): def _append_point(points, sample, xyz, interpolation_samples, meta):
item = (int(sample), *map(float, xyz), int(interpolation_samples)) item = (int(sample), *map(float, xyz), int(interpolation_samples), dict(meta))
if points and item[0] == points[-1][0]: if points and item[0] == points[-1][0]:
points[-1] = item points[-1] = item
elif not points or item[0] > points[-1][0]: elif not points or item[0] > points[-1][0]:
@@ -34,15 +64,17 @@ def _append_point(points, sample, xyz, interpolation_samples):
def _expand_events_dense64(events, total_samples, rate, update_block_samples, def _expand_events_dense64(events, total_samples, rate, update_block_samples,
object_delay_samples, object_index): object_delay_samples, object_index):
if not events: if not events:
return [(0.0, 0.0, 0.0, 0.0, total_samples / float(rate), 0.0)] return [(0.0, 0.0, 0.0, 0.0, total_samples / float(rate), 0.0,
_default_meta())]
block = int(update_block_samples) block = int(update_block_samples)
# 初始位置从成品 sample 0 起有效;合成延迟只作用于后续位置变化。 # 初始位置从成品 sample 0 起有效;合成延迟只作用于后续位置变化。
current = events[0][1] current = events[0][1]
current_meta = events[0][3]
points = [] points = []
_append_point(points, 0, current, 0) _append_point(points, 0, current, 0, current_meta)
for event_index, (coded_start, target, ramp_samples) in enumerate(events[1:], 1): for event_index, (coded_start, target, ramp_samples, meta) in enumerate(events[1:], 1):
start = align_metadata_sample(coded_start + object_delay_samples, block) start = align_metadata_sample(coded_start + object_delay_samples, block)
if start >= total_samples: if start >= total_samples:
break break
@@ -55,12 +87,13 @@ def _expand_events_dense64(events, total_samples, rate, update_block_samples,
# 在运动起点保留上一位置,避免下游把长时间静止段直接连到首个中间点。 # 在运动起点保留上一位置,避免下游把长时间静止段直接连到首个中间点。
if start > points[-1][0]: if start > points[-1][0]:
_append_point(points, start, current, 0) _append_point(points, start, current, 0, current_meta)
ramp = max(0, int(ramp_samples)) ramp = max(0, int(ramp_samples))
if ramp == 0: if ramp == 0:
_append_point(points, start, target, 0) _append_point(points, start, target, 0, meta)
current = target current = target
current_meta = meta
continue continue
end = start + math.ceil(ramp / block) * block end = start + math.ceil(ramp / block) * block
@@ -91,12 +124,14 @@ def _expand_events_dense64(events, total_samples, rate, update_block_samples,
sample = start + elapsed sample = start + elapsed
if sample >= total_samples: if sample >= total_samples:
break break
_append_point(points, sample, position, block) _append_point(points, sample, position, block, meta)
future -= block future -= block
current = target current = target
current_meta = meta
blocks = [] blocks = []
for point_index, (sample, x, y, z, interpolation_samples) in enumerate(points): for point_index, point in enumerate(points):
sample, x, y, z, interpolation_samples, meta = point
end = points[point_index + 1][0] if point_index + 1 < len(points) else total_samples end = points[point_index + 1][0] if point_index + 1 < len(points) else total_samples
duration_samples = max(0, end - sample) duration_samples = max(0, end - sample)
if duration_samples == 0: if duration_samples == 0:
@@ -104,7 +139,7 @@ def _expand_events_dense64(events, total_samples, rate, update_block_samples,
interpolation_samples = min(interpolation_samples, duration_samples) interpolation_samples = min(interpolation_samples, duration_samples)
blocks.append((sample / float(rate), x, y, z, blocks.append((sample / float(rate), x, y, z,
duration_samples / float(rate), duration_samples / float(rate),
interpolation_samples / float(rate))) interpolation_samples / float(rate), meta))
return blocks return blocks
@@ -119,14 +154,16 @@ def _compact_events(events, total_samples, rate, update_block_samples,
``[align(start), align(start) + ramp_duration]``. ``[align(start), align(start) + ramp_duration]``.
""" """
if not events: if not events:
return [(0.0, 0.0, 0.0, 0.0, total_samples / float(rate), 0.0)] return [(0.0, 0.0, 0.0, 0.0, total_samples / float(rate), 0.0,
_default_meta())]
points = [] points = []
current = events[0][1] current = events[0][1]
_append_point(points, 0, current, 0) current_meta = events[0][3]
_append_point(points, 0, current, 0, current_meta)
block = int(update_block_samples) block = int(update_block_samples)
for event_index, (coded_start, target, ramp_samples) in enumerate(events[1:], 1): for event_index, (coded_start, target, ramp_samples, meta) in enumerate(events[1:], 1):
event_start = align_metadata_sample(coded_start + object_delay_samples, block) event_start = align_metadata_sample(coded_start + object_delay_samples, block)
if event_start >= total_samples: if event_start >= total_samples:
break break
@@ -144,7 +181,7 @@ def _compact_events(events, total_samples, rate, update_block_samples,
"previous_sample": points[-1][0]}) "previous_sample": points[-1][0]})
if event_index + 1 < len(events): if event_index + 1 < len(events):
next_coded_start, _, next_ramp_samples = events[event_index + 1] next_coded_start, _, next_ramp_samples = events[event_index + 1][:3]
next_event_start = align_metadata_sample( next_event_start = align_metadata_sample(
next_coded_start + object_delay_samples, block) next_coded_start + object_delay_samples, block)
next_block_start = next_event_start next_block_start = next_event_start
@@ -166,11 +203,13 @@ def _compact_events(events, total_samples, rate, update_block_samples,
if ramp > available: if ramp > available:
block_target = _lerp_xyz(current, target, available / float(ramp)) block_target = _lerp_xyz(current, target, available / float(ramp))
block_interpolation = available block_interpolation = available
_append_point(points, block_start, block_target, block_interpolation) _append_point(points, block_start, block_target, block_interpolation, meta)
current = target current = target
current_meta = meta
blocks = [] blocks = []
for point_index, (sample, x, y, z, interpolation_samples) in enumerate(points): for point_index, point in enumerate(points):
sample, x, y, z, interpolation_samples, meta = point
end = points[point_index + 1][0] if point_index + 1 < len(points) else total_samples end = points[point_index + 1][0] if point_index + 1 < len(points) else total_samples
duration_samples = max(0, end - sample) duration_samples = max(0, end - sample)
if duration_samples == 0: if duration_samples == 0:
@@ -178,7 +217,7 @@ def _compact_events(events, total_samples, rate, update_block_samples,
interpolation_samples = min(interpolation_samples, duration_samples) interpolation_samples = min(interpolation_samples, duration_samples)
blocks.append((sample / float(rate), x, y, z, blocks.append((sample / float(rate), x, y, z,
duration_samples / float(rate), duration_samples / float(rate),
interpolation_samples / float(rate))) interpolation_samples / float(rate), meta))
return blocks return blocks
@@ -192,6 +231,23 @@ def _expand_events(events, total_samples, rate, update_block_samples,
object_delay_samples, object_index) object_delay_samples, object_index)
raise ValueError(f"未知 trajectory_mode: {trajectory_mode}") raise ValueError(f"未知 trajectory_mode: {trajectory_mode}")
def read_atmos_meta(index, frames=None, max_scan=64):
"""从 OAMD payload 读取 trim/warp 配置。
trim element 描述整段节目的校准意图,跨帧恒定,因此扫到第一帧含 trim 的
payload 即可。返回 ``None`` 表示该节目没有 trim element。
"""
rows = index.rows if frames is None else frames
for row in rows[:max_scan]:
subs = index.subpayloads(row)
if not subs or 11 not in subs:
continue
meta = parse_trim(subs[11])
if meta is not None:
return meta
return None
def build_adm_tracks(index, frames=None, rate=48000, frame_samples=1536, def build_adm_tracks(index, frames=None, rate=48000, frame_samples=1536,
update_block_samples=METADATA_BLOCK_SAMPLES, update_block_samples=METADATA_BLOCK_SAMPLES,
object_delay_samples=1473, trajectory_mode="compact"): object_delay_samples=1473, trajectory_mode="compact"):
@@ -209,13 +265,16 @@ def build_adm_tracks(index, frames=None, rate=48000, frame_samples=1536,
state = JocFieldState() state = JocFieldState()
events = [[] for _ in range(15)] events = [[] for _ in range(15)]
previous = [None] * 15 previous = [None] * 15
meta_state = [_default_meta() for _ in range(15)]
for seq, row in enumerate(frames): for seq, row in enumerate(frames):
subs = index.subpayloads(row) subs = index.subpayloads(row)
timing = None timing = None
full = None
if subs is not None and 11 in subs: if subs is not None and 11 in subs:
timing = frame_update(subs[11]) timing = frame_update(subs[11])
state.apply(timing["values"]) state.apply(timing["values"])
full = frame_state(subs[11])
event_sample = seq * frame_samples event_sample = seq * frame_samples
ramp_samples = 0 ramp_samples = 0
@@ -228,9 +287,12 @@ def build_adm_tracks(index, frames=None, rate=48000, frame_samples=1536,
q = state.q q = state.q
for obj in range(1, 16): for obj in range(1, 16):
xyz = q_to_adm_xyz(q[(obj, "q1")], q[(obj, "q2")], q[(obj, "q3")]) xyz = q_to_adm_xyz(q[(obj, "q1")], q[(obj, "q2")], q[(obj, "q3")])
if previous[obj - 1] != xyz: if full is not None and obj < len(full["objects"]):
events[obj - 1].append((event_sample, xyz, ramp_samples)) _apply_state(meta_state[obj - 1], full["objects"][obj])
previous[obj - 1] = xyz meta = dict(meta_state[obj - 1])
if previous[obj - 1] != (xyz, meta):
events[obj - 1].append((event_sample, xyz, ramp_samples, meta))
previous[obj - 1] = (xyz, meta)
total_samples = len(frames) * frame_samples total_samples = len(frames) * frame_samples
# bed 对象在 OAMD 里没有位置字段:位置由声明的 bed 布局决定,整片静止在该方位。 # bed 对象在 OAMD 里没有位置字段:位置由声明的 bed 布局决定,整片静止在该方位。
@@ -241,7 +303,7 @@ def build_adm_tracks(index, frames=None, rate=48000, frame_samples=1536,
if fixed is not None: if fixed is not None:
tracks.append((f"JOC_Object_{obj}", tracks.append((f"JOC_Object_{obj}",
[(0.0, fixed[0], fixed[1], fixed[2], [(0.0, fixed[0], fixed[1], fixed[2],
total_samples / float(rate), 0.0)])) total_samples / float(rate), 0.0, _default_meta())]))
continue continue
tracks.append( tracks.append(
(f"JOC_Object_{obj}", (f"JOC_Object_{obj}",
+3 -5
View File
@@ -16,7 +16,7 @@ class DistanceState:
class ReferenceDistanceProfileV1: class ReferenceDistanceProfileV1:
"""Reference behavior, not a claim about any external public standard.""" """Near/Mid/Far distance presets with energy-normalized dry/room mixing."""
DISTANCE_M = { DISTANCE_M = {
"near": 1.00000465, "near": 1.00000465,
@@ -24,10 +24,8 @@ class ReferenceDistanceProfileV1:
"far": 6.40177584, "far": 6.40177584,
} }
MINIMUM_DISTANCE_M = 0.10 MINIMUM_DISTANCE_M = 0.10
# Distance profiles in the reference renderer are presentation presets, # Distance presets use an energy-normalized dry/room crossfade without
# not an instruction to attenuate already-authored programme PCM by 1/r. # applying 1/r attenuation to the programme PCM.
# Use an energy-normalized dry/room crossfade instead. The coefficient is
# an explicit project calibration target.
ROOM_ENERGY_COUPLING_PER_M2 = 0.01318359375 ROOM_ENERGY_COUPLING_PER_M2 = 0.01318359375
PUBLIC_ROOM_CALIBRATION_GAIN = 1.4 PUBLIC_ROOM_CALIBRATION_GAIN = 1.4
# Public, project-owned room coupling; it is not a SOFA or Dolby constant. # Public, project-owned room coupling; it is not a SOFA or Dolby constant.
+315 -71
View File
@@ -1,11 +1,14 @@
"""把 E-AC-3 核心 PCM 和 JOC 元数据重建为 LFE 加 15 路对象 PCM。 """把 E-AC-3 核心 PCM 和 JOC 元数据重建为 LFE 加 15 路对象 PCM。
管线(每帧): 管线(每帧):
核心 PCM(N=5 或 7 路全频段通道,按 k 查表)→ QMF 分析 x → 填充器 z = Σ m·x(每对象) 核心 PCM(N=5 或 7 路全频段通道,按 k 查表)→ QMF 分析 x → 对象矩阵 z = Σ m·x
→ 对象时域组装(前步、64 点 FFT、旋转、合成窗)→ ×16 + clamp → 对象时域组装(前步、64 点 FFT、旋转、合成窗)→ ×16 + clamp
→ 对象波形 × joc_clipgain → LFE 专用 1217-sample 环形延迟(逐点乘 clipgain 斜坡)
+ LFE 专用 1217-sample 环形延迟
→ 16ch(ch0 = LFE, ch1-15 = 15 对象)→ ×output_scale → 16ch(ch0 = LFE, ch1-15 = 15 对象)→ ×output_scale
无元数据帧改为 8 路核心 PCM identity 直通(槽 0..7,每路延迟 1217),槽 8..15 为零。
跨帧状态:seq 状态机(state/expected/无效帧计数)、每对象的 prev 矩阵与存在标记、
QMF 分析与合成状态、LFE 延迟环、直通渲染的 8 路核心 PCM 历史。
output_scale: output_scale:
默认 1.0(0 dB)。用户选择的 dB 在命令行转换为 float32 系数;该系数 默认 1.0(0 dB)。用户选择的 dB 在命令行转换为 float32 系数;该系数
@@ -19,29 +22,57 @@ from joc_qmf import (N, QMF5_WINDOW, qmf_analysis_frame,
from evo_unpack import unpack_evolution from evo_unpack import unpack_evolution
CORE_CHANNELS = [0, 1, 2, 4, 5] # L R C Ls Rs(EAC3 5.1 核心顺序) CORE_CHANNELS = [0, 1, 2, 4, 5] # L R C Ls Rs(EAC3 5.1 核心顺序)
FRAME_SAMPLES = 1536
MAX_OBJECTS = 15
MAX_CORE_CHANNELS = 7
CORE_OUT_CHANNELS = 8
LFE_SLOT = 3
LFE_DELAY = 1217
SEQ_MODULUS = 1024
# 没有 JOC 元数据的帧:状态机短路分支使用这个状态码。
STATUS_NO_METADATA = 8
# 渲染层给出的“核心声道数变化”状态码。
STATUS_CORE_CHANNEL_CHANGE = 9
def next_seq(value):
"""seq 的推进规则:1023 之后回到 1。"""
return 1 if value == SEQ_MODULUS - 1 else value + 1
class JocRenderer: class JocRenderer:
"""E-AC-3 JOC 帧渲染器。状态跨帧保持(FIFO/插值 prev/合成状态)。""" """E-AC-3 JOC 帧渲染器。状态跨帧保持(FIFO/插值 prev/合成状态/状态机)。"""
def __init__(self, output_scale=1.0): def __init__(self, output_scale=1.0):
# 成品增益明确按 float32 运算;默认值为 1.0。 # 成品增益明确按 float32 运算;默认值为 1.0。
self.output_scale = np.float32(output_scale) self.output_scale = np.float32(output_scale)
self._prev = {} # 每对象插值 prev(m 的帧末 sb 值) # 每对象插值 prev:上一帧发布的末时槽矩阵与“prev 存在”标记;槽位固定 15 个。
self._synthesis_state = np.zeros((15, 640), dtype=np.float64) self._prev_mtx = np.zeros((MAX_OBJECTS, MAX_CORE_CHANNELS, N), dtype=np.float64)
self._prev_present = np.zeros(MAX_OBJECTS, dtype=bool)
self._synthesis_state = np.zeros((MAX_OBJECTS, 640), dtype=np.float64)
# 分析 QMF:最多七路,各保留 9×64 FIFO。L/R/C 与 N=7 时的第 6、7 路 # 分析 QMF:最多七路,各保留 9×64 FIFO。L/R/C 与 N=7 时的第 6、7 路
# (额外那一对)在这里做 10 时隙延迟;Ls/Rs 的同一延迟在环绕通路内完成。 # (额外那一对)在这里做 10 时隙延迟;Ls/Rs 的同一延迟在环绕通路内完成。
self._analysis_fifo = np.zeros((7, 9, 64), dtype=np.float64) self._analysis_fifo = np.zeros((MAX_CORE_CHANNELS, 9, 64), dtype=np.float64)
self._analysis_delay = np.zeros((5, 10, 64), dtype=np.float32) self._analysis_delay = np.zeros((5, 10, 64), dtype=np.float32)
# 哨兵:复位后(或整条流的第一帧)直接用新标量,不做帧内过渡。 # 哨兵:复位后(或整条流的第一帧)直接用新标量,不做帧内过渡。
self._analysis_phase = np.float32(-0.5) self._analysis_phase = np.float32(-0.5)
self._surround_qmf_delay = np.zeros((2, 10, 64), dtype=np.complex128) self._surround_qmf_delay = np.zeros((2, 10, 64), dtype=np.complex128)
self._surround_dc_hist = np.zeros((2, 20), dtype=np.complex128) self._surround_dc_hist = np.zeros((2, 20), dtype=np.complex128)
# ch0/LFE 使用循环历史,读指针恒落后写指针 1217 个采样。 # ch0/LFE 使用循环历史,读指针恒落后写指针 1217 个采样。
# phase=1/16 与输出端 *16 抵消,净效果是 LFE 延迟。 # 逐点增益斜坡在写入延迟环之前乘上,读出后不再乘任何增益。
self._lfe_delay = np.zeros(1217, dtype=np.float64) self._lfe_delay = np.zeros(LFE_DELAY, dtype=np.float64)
self._last_x = None # 仅供逐槽诊断读取,不参与状态推进 self._lfe_clipgain = None # LFE 增益斜坡的起点;None = 还没有历史
self._rot = None # 每带旋转表 # 直通渲染的 8 路核心 PCM 延迟历史:保存上一帧输入。
self._core_hist = np.zeros((CORE_OUT_CHANNELS, FRAME_SAMPLES), dtype=np.float64)
# seq 状态机与逐帧分派所需的跨帧状态。
self._state = 0
self._expected = 0
self._invalid_count = 0
self._last_frame_valid = False
self._n_channels_prev = None
self._config_idx = 3
self._last_x = None # 仅供逐槽诊断读取,不参与状态推进
self._rot = None # 每带旋转表
def held_analysis_scalar(self): def held_analysis_scalar(self):
"""当前分析端标量;还没有过 JOC 帧时用结构常数 1/16。""" """当前分析端标量;还没有过 JOC 帧时用结构常数 1/16。"""
@@ -50,21 +81,27 @@ class JocRenderer:
return np.float32(0.0625) return np.float32(0.0625)
return value return value
def held_clipgain(self):
"""当前生效的完整 clipgain;没有历史时为 1.0。"""
return float(self.held_analysis_scalar()) * 16.0
@staticmethod @staticmethod
def decode_payload(payload_bytes): def decode_payload(payload_bytes):
"""id14 载荷 → (parse 结果, mix_q, mix_dq)。""" """id14 载荷 → (parse 结果, mix_q, mix_dq)。"""
subs, _ = unpack_evolution(payload_bytes, loose=True) subs, _ = unpack_evolution(payload_bytes, loose=True)
out = parse_joc(subs[14]) return JocRenderer.decode_subpayloads(subs)
mix_q = diff_decode(out)
mix_dq = dequantize(out, mix_q)
return out, mix_q, mix_dq
@staticmethod @staticmethod
def decode_subpayloads(subs): def decode_subpayloads(subs):
"""已解析 EMDF 子载荷 → (parse 结果, mix_q, mix_dq)。""" """已解析 EMDF 子载荷 → (parse 结果, mix_q, mix_dq)。
整帧状态不是 0 时只返回解析结果:这一帧按无效帧处理,不做矩阵还原。
"""
if 14 not in subs: if 14 not in subs:
raise ValueError("EMDF 缺少 ID14/JOC") raise ValueError("EMDF 缺少 ID14/JOC")
out = parse_joc(subs[14]) out = parse_joc(subs[14])
if out["status"] != 0:
return out, {}, {}
mix_q = diff_decode(out) mix_q = diff_decode(out)
mix_dq = dequantize(out, mix_q) mix_dq = dequantize(out, mix_q)
return out, mix_q, mix_dq return out, mix_q, mix_dq
@@ -80,13 +117,13 @@ class JocRenderer:
""" """
pcm = np.asarray(bed_pcm, dtype=np.float32) pcm = np.asarray(bed_pcm, dtype=np.float32)
n_channels = pcm.shape[0] if pcm.ndim == 2 else 0 n_channels = pcm.shape[0] if pcm.ndim == 2 else 0
if n_channels not in (5, 7) or pcm.shape != (n_channels, 1536): if n_channels not in (5, 7) or pcm.shape != (n_channels, FRAME_SAMPLES):
raise ValueError(f"核心帧应为 (5,1536) 或 (7,1536),实际 {pcm.shape}") raise ValueError(f"核心帧应为 (5,1536) 或 (7,1536),实际 {pcm.shape}")
new_phase = np.float32(phase_new) new_phase = np.float32(phase_new)
old_phase = self._analysis_phase old_phase = self._analysis_phase
if old_phase <= 0.0 and old_phase != 0.0: # 哨兵:首帧/复位后不做过过渡 if old_phase <= 0.0 and old_phase != 0.0: # 哨兵:首帧/复位后不做过过渡
old_phase = new_phase old_phase = new_phase
gains = np.full(1536, new_phase, dtype=np.float32) gains = np.full(FRAME_SAMPLES, new_phase, dtype=np.float32)
if old_phase != new_phase: if old_phase != new_phase:
step = np.float32((new_phase - old_phase) / np.float32(256.0)) step = np.float32((new_phase - old_phase) / np.float32(256.0))
gains[:256] = old_phase + np.arange(256, dtype=np.float32) * step gains[:256] = old_phase + np.arange(256, dtype=np.float32) * step
@@ -110,24 +147,117 @@ class JocRenderer:
apply_dc_filter=apply_dc_filter, rotate=rotate) apply_dc_filter=apply_dc_filter, rotate=rotate)
return x return x
def _clear_unused_channels(self, n_channels):
"""核心声道数变化后,只清除新布局中未使用声道的历史状态。"""
if n_channels >= MAX_CORE_CHANNELS:
return
self._analysis_fifo[n_channels:] = 0.0
self._prev_mtx[:, n_channels:, :] = 0.0
if n_channels == 5:
# N=5 不用额外那一对,10 时隙延迟线 3、4 号槽不再推进。
self._analysis_delay[3:] = 0.0
def _maybe_clear_lfe_ring(self, status):
"""声明声道数与实际核心声道数失配(本帧 status=9)时清空 LFE 环。"""
if status == STATUS_CORE_CHANNEL_CHANGE:
self._lfe_delay.fill(0.0)
def _advance_state(self, status, seq):
"""按 seq 推进状态机,返回进入本帧前的 ``state``。
status:0 正常;2..7 解析层无效(7 = ID14 载荷在 joc_data 内提前结束);
8 无元数据;9 核心声道数变化。
"""
pre_state = self._state
valid = status in (0, STATUS_CORE_CHANNEL_CHANGE)
if status == STATUS_NO_METADATA and pre_state != 0:
# 无元数据帧不推进期望、不比较 seq。
self._state = 3
return pre_state
self._expected = next_seq(self._expected)
if valid and seq == 0 and pre_state != 0:
# seq 回到 0:节目重新开始,硬切并清空对象历史。
self._state = 4
return pre_state
state = self._state
if state == 0:
if valid:
state = 1
self._expected = seq
elif state == 1:
if valid:
if seq != self._expected:
state = 4
else:
state = 2
self._invalid_count = 1
elif state == 2:
if valid:
state = 1
self._expected = seq
elif self._invalid_count == 5:
state = 3
else:
self._invalid_count += 1
elif state == 3:
if valid:
state = 1
self._expected = seq
else:
self._invalid_count += 1
elif state == 4:
if valid:
state = 1
self._expected = seq
else:
state = 2
self._invalid_count = 1
self._state = state
if status == STATUS_CORE_CHANNEL_CHANGE:
self._state = 4
self._last_frame_valid = valid
return pre_state
def object_z(self, out, mix_dq, x): def object_z(self, out, mix_dq, x):
"""计算 ``z[obj] = Σ_ch m[ch,sb,ts]·x[ch,sb,ts]``。""" """计算恢复正常渲染时的 ``z[obj] = Σ_ch m[ch,sb,ts]·x[ch,sb,ts]``。
对象槽固定 15 个:本帧声明范围内 present=1 的对象按插值规则生成矩阵;
present 由 1 变 0(或落在声明范围之外)的对象按“present=1 且矩阵全零”
继续发布,直到它重新 present=1;从未出现过的槽输出零。
"""
n_channels = x.shape[0]
objs = out["objs"]
z_all = {} z_all = {}
new_prev = {} for obj in range(MAX_OBJECTS):
for obj, o in enumerate(out["objs"]): present = obj < len(objs) and objs[obj]["present"]
if not o["present"]: if present:
continue prev = self._prev_mtx[obj][:n_channels] if self._prev_present[obj] else None
dq = mix_dq[obj] mtx = interp_matrix(objs[obj], mix_dq[obj], prev)
prev = self._prev.get(obj) z_all[obj] = np.sum(x * mtx, axis=0)
if prev is None: self._prev_mtx[obj][:n_channels] = mtx[:, :, -1]
prev = np.zeros((out["n_channels"], N), dtype=np.float64) self._prev_present[obj] = True
m = interp_matrix(o, dq, prev) # [ch][sb][ts](内部按 o["n_bands"] 映射) elif self._prev_present[obj]:
z = np.sum(x * m, axis=0) # [sb][ts](复) z_all[obj] = np.zeros((N, 24), dtype=np.complex128)
z_all[obj] = z self._prev_mtx[obj][:n_channels] = 0.0
new_prev[obj] = m[:, :, -1] self._prev_present[obj] = True
self._prev.update(new_prev) else:
z_all[obj] = np.zeros((N, 24), dtype=np.complex128)
return z_all return z_all
def _hold_objects(self, x):
"""保持渲染:24 个时隙全部使用上一帧发布的矩阵,不更新 prev。"""
n_channels = x.shape[0]
z_all = {}
for obj in range(MAX_OBJECTS):
z_all[obj] = np.sum(x * self._prev_mtx[obj][:n_channels][:, :, None], axis=0)
return z_all
@staticmethod
def _zero_objects():
return {obj: np.zeros((N, 24), dtype=np.complex128)
for obj in range(MAX_OBJECTS)}
def _rot_table_86840(self): def _rot_table_86840(self):
"""生成 ``θ=πk/128`` 的 ``0.5·(sin θ, cos θ)`` 旋转表。""" """生成 ``θ=πk/128`` 的 ``0.5·(sin θ, cos θ)`` 旋转表。"""
if self._rot is None: if self._rot is None:
@@ -218,7 +348,7 @@ class JocRenderer:
z = np.stack([z_all[obj] for obj in object_ids], axis=0) z = np.stack([z_all[obj] for obj in object_ids], axis=0)
state = self._synthesis_state[object_ids].copy() state = self._synthesis_state[object_ids].copy()
rot868 = self._rot_table_86840() rot868 = self._rot_table_86840()
pcm = np.zeros((len(object_ids), 1536), dtype=np.float64) pcm = np.zeros((len(object_ids), FRAME_SAMPLES), dtype=np.float64)
for tsg in range(6): for tsg in range(6):
ring = np.zeros((len(object_ids), 512), dtype=np.float64) ring = np.zeros((len(object_ids), 512), dtype=np.float64)
chunk = z[:, :, tsg * 4:tsg * 4 + 4].transpose(0, 2, 1) chunk = z[:, :, tsg * 4:tsg * 4 + 4].transpose(0, 2, 1)
@@ -242,39 +372,158 @@ class JocRenderer:
"""返回逆 QMF 使用的 640 项有效合成窗表。""" """返回逆 QMF 使用的 640 项有效合成窗表。"""
return QMF5_WINDOW return QMF5_WINDOW
def decode_lfe(self, lfe_pcm): def decode_lfe(self, lfe_pcm, clipgain=None):
"""将核心 ch3/LFE 经过跨帧 1217-sample 延迟后输出到 ch0。""" """核心 ch3/LFE 乘逐点增益斜坡,再经跨帧 1217-sample 延迟输出到 ch0。
clipgain 为本帧完整增益 G_new;None 表示本帧没有新值(沿用当前值)。
斜坡:n<256 时 G_old + n·(G_new−G_old)/256,n≥256 时 G_new;
首帧(没有历史)时 G_old = G_new。
"""
current = np.asarray(lfe_pcm, dtype=np.float64) current = np.asarray(lfe_pcm, dtype=np.float64)
if current.shape != (1536,): if current.shape != (FRAME_SAMPLES,):
raise ValueError(f"LFE 帧应为 (1536,),实际 {current.shape}") raise ValueError(f"LFE 帧应为 (1536,),实际 {current.shape}")
delayed = np.concatenate([self._lfe_delay, current]) new_gain = self.held_clipgain() if clipgain is None else float(clipgain)
out = np.clip(delayed[:1536], -1.0, 1.0) old_gain = new_gain if self._lfe_clipgain is None else self._lfe_clipgain
self._lfe_delay = delayed[-1217:].copy() gains = np.full(FRAME_SAMPLES, new_gain, dtype=np.float64)
if old_gain != new_gain:
gains[:256] = old_gain + np.arange(256, dtype=np.float64) * (
(new_gain - old_gain) / 256.0)
self._lfe_clipgain = new_gain
delayed = np.concatenate([self._lfe_delay, current * gains])
out = np.clip(delayed[:FRAME_SAMPLES], -1.0, 1.0)
self._lfe_delay = delayed[-LFE_DELAY:].copy()
return out return out
def render_missing_frame(self, bed_pcm, lfe_pcm=None, config_idx=3): @staticmethod
"""没有 JOC 元数据的音频帧:对象通道静音,只保留 LFE 旁通。 def _validate_bed(bed_pcm):
pcm = np.asarray(bed_pcm, dtype=np.float32)
n_channels = pcm.shape[0] if pcm.ndim == 2 else 0
if n_channels not in (5, 7) or pcm.shape != (n_channels, FRAME_SAMPLES):
raise ValueError(f"核心帧应为 (5,1536) 或 (7,1536),实际 {pcm.shape}")
return pcm
QMF 分析与对象合成状态照常推进,否则后续帧的时间轴会整体少一格。 @staticmethod
""" def _core8(core, lfe):
sigma = config_idx in SIGMA_DMX_CONFIGS """JOC 输入序 5/7 路(不含 LFE)→ 8 平面核心序 L,R,C,LFE,Ls,Rs,(Lrs,Rrs)。"""
# 本帧没有 JOC 元数据,分析端标量保持上一帧的值(槽位不被改写)。 out = np.zeros((CORE_OUT_CHANNELS, FRAME_SAMPLES), dtype=np.float64)
x = self.qmf_x(bed_pcm, phase_new=self.held_analysis_scalar(), out[0:3] = core[0:3]
apply_dc_filter=sigma, rotate=sigma) out[LFE_SLOT] = lfe
rest = core.shape[0] - 3
if rest > 0:
out[4:4 + rest] = core[3:]
return out
@staticmethod
def _delayed_core(hist, current):
"""直通渲染:8 路核心 PCM 各延迟 1217 个采样。"""
out = np.empty((CORE_OUT_CHANNELS, FRAME_SAMPLES), dtype=np.float64)
out[:, :LFE_DELAY] = hist[:, FRAME_SAMPLES - LFE_DELAY:]
out[:, LFE_DELAY:] = current[:, :FRAME_SAMPLES - LFE_DELAY]
return out
def _render_frame(self, *, status, seq, out, mix_dq, bed_pcm, lfe_pcm,
config_idx=None):
"""每帧执行一次状态机并按结果分派渲染。"""
pcm = self._validate_bed(bed_pcm)
n_channels = pcm.shape[0]
if self._n_channels_prev is not None and n_channels != self._n_channels_prev:
self._clear_unused_channels(n_channels)
self._advance_state(status, seq)
state = self._state
self._maybe_clear_lfe_ring(status)
# 兼容性说明(status 8 无元数据帧的状态机改动):
# 变更前:status 8 无条件走 passthrough,流中途也一样。passthrough 用
# 延迟后的 8 路 core 平面替换对象渲染,对象合成在下一帧从静音重新开始,
# 于是每个无元数据帧都会在帧界产生一次台阶,元数据恢复时再产生一次;
# 对不够标准的编码器(JOC EMDF 容器写得不好)这听起来就是噼啪声。
# 变更后:status 8 和其他无效状态一样交给状态机——state 0(还没渲染过
# 有效帧,即普通 E-AC-3 或 JOC 首帧之前)仍然 passthrough(核心 PCM
# 直通本来就是为这种流准备的);state 3(流中途)走 hold,沿用上一帧
# 的对象矩阵继续渲染,床与对象都保持连续。
# 说明:这是为兼容性做的取舍,不代表规范解释一定如此——"无元数据"也可以
# 被论证为就是 passthrough 的情形;若要回到参考行为,恢复这里的无条件
# passthrough 分支即可。
if status == STATUS_NO_METADATA and state == 0:
mode = "passthrough"
elif state == 4:
mode = "hard_cut"
elif state in (2, 3):
mode = "hold"
elif state == 1 and status == 0:
mode = "normal"
else:
mode = "passthrough"
if out is not None and out.get("n_channels") is not None:
config = out["dmx_config_idx"]
self._config_idx = config
elif config_idx is not None:
config = config_idx
self._config_idx = config
else:
config = self._config_idx
sigma = config in SIGMA_DMX_CONFIGS
valid_metadata = (out is not None and out.get("status") == 0
and out.get("n_channels") is not None)
if mode in ("normal", "hard_cut") and valid_metadata:
clipgain = float(out["clipgain"])
phase_new = 0.0625 * clipgain
else:
clipgain = self.held_clipgain()
phase_new = self.held_analysis_scalar()
lfe = (np.zeros(FRAME_SAMPLES, dtype=np.float64)
if lfe_pcm is None else np.asarray(lfe_pcm, dtype=np.float64))
if lfe.shape != (FRAME_SAMPLES,):
raise ValueError(f"LFE 帧应为 (1536,),实际 {lfe.shape}")
core8 = self._core8(pcm, lfe)
x = self.qmf_x(pcm, phase_new=phase_new, apply_dc_filter=sigma, rotate=sigma)
self._last_x = x.copy() self._last_x = x.copy()
z_all = {obj: np.zeros((x.shape[1], x.shape[2]), dtype=np.complex128) pcm16 = np.zeros((MAX_OBJECTS + 1, FRAME_SAMPLES), dtype=np.float64)
for obj in range(15)}
self.dll_synth_objects(z_all) if mode == "normal":
pcm16 = np.zeros((16, 1536), dtype=np.float64) z_all = self.object_z(out, mix_dq, x)
if lfe_pcm is not None: objs_pcm = self.dll_synth_objects(z_all)
pcm16[0] = self.decode_lfe(lfe_pcm) pcm16[0] = self.decode_lfe(lfe, clipgain)
for obj, pcm_obj in objs_pcm.items():
pcm16[obj + 1] = pcm_obj
elif mode == "hold":
z_all = self._hold_objects(x)
objs_pcm = self.dll_synth_objects(z_all)
pcm16[0] = self.decode_lfe(lfe, clipgain)
for obj, pcm_obj in objs_pcm.items():
pcm16[obj + 1] = pcm_obj
elif mode == "hard_cut":
# 全部对象矩阵清零,保留“prev 存在”标记,下一有效帧从 0 淡入。
z_all = self._zero_objects()
self.dll_synth_objects(z_all)
self.decode_lfe(lfe, clipgain) # 环照常写入并推进写游标
self._prev_present[:] = True
self._prev_mtx[:] = 0.0
else: # passthrough
z_all = self._zero_objects()
self.dll_synth_objects(z_all)
self.decode_lfe(lfe, clipgain)
pcm16[:CORE_OUT_CHANNELS] = self._delayed_core(self._core_hist, core8)
self._core_hist[:] = core8
self._n_channels_prev = n_channels
pcm16_f32 = np.asarray(pcm16, dtype=np.float32) pcm16_f32 = np.asarray(pcm16, dtype=np.float32)
return np.multiply(pcm16_f32, self.output_scale, dtype=np.float32), z_all return np.multiply(pcm16_f32, self.output_scale, dtype=np.float32), z_all
def render_missing_frame(self, bed_pcm, lfe_pcm=None, config_idx=3):
"""没有 JOC 元数据的音频帧:8 路核心 PCM identity 直通(每路延迟 1217 采样)。"""
return self._render_frame(status=STATUS_NO_METADATA, seq=None,
out=None, mix_dq=None,
bed_pcm=bed_pcm, lfe_pcm=lfe_pcm,
config_idx=config_idx)
def render_frame(self, payload_bytes, bed_pcm, lfe_pcm=None): def render_frame(self, payload_bytes, bed_pcm, lfe_pcm=None):
"""payload_bytes = evolution 载荷(含 id14 JOC);bed_pcm = N×1536 核心 PCM。 """payload_bytes = evolution 载荷(含 id14 JOC);bed_pcm = N×1536 核心 PCM。
lfe_pcm = 1536 核心 LFE;提供时生成 ch0,省略时 ch0 静音
(保留给只验证对象/z 链的诊断脚本)。
返回 (pcm16, z_all):pcm16 = 16×1536(ch0 LFE + 15 对象)f32, 返回 (pcm16, z_all):pcm16 = 16×1536(ch0 LFE + 15 对象)f32,
已用 FP32 系数乘 output_scale。""" 已用 FP32 系数乘 output_scale。"""
subs, _ = unpack_evolution(payload_bytes, loose=True) subs, _ = unpack_evolution(payload_bytes, loose=True)
@@ -283,19 +532,14 @@ class JocRenderer:
def render_subpayloads(self, subs, bed_pcm, lfe_pcm=None): def render_subpayloads(self, subs, bed_pcm, lfe_pcm=None):
"""以已拆出的 EMDF payload 字典渲染一帧,避免绑定 transport 容器。""" """以已拆出的 EMDF payload 字典渲染一帧,避免绑定 transport 容器。"""
out, mix_q, mix_dq = self.decode_subpayloads(subs) out, mix_q, mix_dq = self.decode_subpayloads(subs)
# σ(k) 同时门控 -j 旋转与 band-0 FIR(规格 §4)。 pcm = self._validate_bed(bed_pcm)
sigma = out["dmx_config_idx"] in SIGMA_DMX_CONFIGS status = out["status"]
x = self.qmf_x(bed_pcm, phase_new=0.0625 * out["clipgain"], if (status == 0 and out["n_channels"] is not None
apply_dc_filter=sigma, rotate=sigma) and self._n_channels_prev is not None
self._last_x = x.copy() and out["n_channels"] != self._n_channels_prev):
z_all = self.object_z(out, mix_dq, x) status = STATUS_CORE_CHANNEL_CHANGE
# joc_clipgain 已在分析端进入 qmf_x,与用户 output_scale 分离。 config_idx = (out["dmx_config_idx"] if out["n_channels"] is not None else None)
objs_pcm = self.dll_synth_objects(z_all) return self._render_frame(status=status, seq=out.get("seq_count_bits"),
pcm16 = np.zeros((16, 1536), dtype=np.float64) out=out, mix_dq=mix_dq,
pcm16[0] = self.decode_lfe(lfe_pcm) if lfe_pcm is not None else 0.0 bed_pcm=pcm, lfe_pcm=lfe_pcm,
for obj, p in objs_pcm.items(): config_idx=config_idx)
pcm16[obj + 1] = p
pcm16_f32 = np.asarray(pcm16, dtype=np.float32)
return np.multiply(pcm16_f32, self.output_scale, dtype=np.float32), z_all
+2 -4
View File
@@ -5,12 +5,10 @@ The Rosella renderer can run either bank:
``PUBLIC_FILTERBANK`` (1) ``PUBLIC_FILTERBANK`` (1)
The shared public bank of ``rosella_filterbank.py``: a 13-tap hybrid The shared public bank of ``rosella_filterbank.py``: a 13-tap hybrid
analysis, a 6-slot high-band delay and 961 samples of analysis/synthesis analysis, a 6-slot high-band delay and 961 samples of analysis/synthesis
latency. This is the bank the reference implementation has always used. latency.
``CAPTURED_FILTERBANK`` (2) ``CAPTURED_FILTERBANK`` (2)
The bank the reference headphone renderer actually runs, captured from it: An 11-tap hybrid analysis, a 3-slot high-band delay and 448 samples of latency.
an 11-tap hybrid analysis, a 3-slot high-band delay and 448 samples of
latency.
Both banks expose the same four stateful stages, so the choice changes the Both banks expose the same four stateful stages, so the choice changes the
coefficients, the high-band delay, the history layout and the latency together. coefficients, the high-band delay, the history layout and the latency together.
+8 -4
View File
@@ -15,7 +15,7 @@ from pathlib import Path
import numpy as np import numpy as np
from bed_layout import bed_object_positions from bed_layout import bed_object_positions
from binaural_metadata import OamdPositionTimeline from binaural_metadata import FixedPositionTimeline, OamdPositionTimeline
from rosella_bank import (DEFAULT_FILTERBANK, FILTERBANK_CHOICES, FILTERBANK_NAMES, from rosella_bank import (DEFAULT_FILTERBANK, FILTERBANK_CHOICES, FILTERBANK_NAMES,
RosellaBank) RosellaBank)
from rosella_core import RosellaRenderer from rosella_core import RosellaRenderer
@@ -68,7 +68,8 @@ class RosellaBinauralRenderer:
output_gain: float = 1.0, output_gain: float = 1.0,
chunk_frames: int = 64, chunk_frames: int = 64,
room_impulse_slots: int = 4096, room_impulse_slots: int = 4096,
bed_channels=None): bed_channels=None,
fixed_positions=None):
if mode not in BINAURAL_PROFILE_NAMES: if mode not in BINAURAL_PROFILE_NAMES:
raise ValueError("binaural mode must be near, mid, or far") raise ValueError("binaural mode must be near, mid, or far")
if filterbank not in FILTERBANK_CHOICES: if filterbank not in FILTERBANK_CHOICES:
@@ -117,8 +118,11 @@ class RosellaBinauralRenderer:
room_impulse_slots=room_impulse_slots) room_impulse_slots=room_impulse_slots)
self.hybrid_synthesis = self.bank.hybrid_synthesis self.hybrid_synthesis = self.bank.hybrid_synthesis
self.qmf_synthesis = self.bank.qmf_synthesis self.qmf_synthesis = self.bank.qmf_synthesis
self.timeline = OamdPositionTimeline( if fixed_positions is not None:
15, bed_positions=bed_object_positions(bed_channels or ())) self.timeline = FixedPositionTimeline(fixed_positions)
else:
self.timeline = OamdPositionTimeline(
15, bed_positions=bed_object_positions(bed_channels or ()))
self._input_buffer = np.empty( self._input_buffer = np.empty(
(self.chunk_samples, SOURCE_CHANNELS), dtype=np.float64) (self.chunk_samples, SOURCE_CHANNELS), dtype=np.float64)
+6 -14
View File
@@ -1,16 +1,8 @@
"""Captured Rosella 64-QMF / 77-hybrid filterbank. """Rosella 64-QMF / 77-hybrid filterbank with an 11-tap hybrid analysis.
The Rosella headphone renderer selects its analysis and synthesis callbacks Uses a 640-coefficient QMF prototype, a 3-slot high-band delay and a 448-sample
during preparation, after initialization. The callbacks it actually runs are analysis/synthesis latency. Tables in ``data/rosella_captured_kernels.npz`` are
this bank: a 640-coefficient QMF prototype, an 11-tap hybrid analysis with a loaded as float64/complex128; processing state uses the same precision.
3-slot high-band delay, and a 448-sample analysis/synthesis latency. The shared
public bank lives in ``rosella_filterbank.py`` and differs in every one of those
parameters (13-tap hybrid, 6-slot delay, 961-sample latency).
The tables are the float32 values the C++ core compiles in as
``src/hrtf/rosella_filterbank_tables.h``; ``data/rosella_captured_kernels.npz``
holds the same numbers. State is float64/complex128, like the rest of the
reference implementation.
""" """
from __future__ import annotations from __future__ import annotations
@@ -30,7 +22,7 @@ HYBRID_DELAY = 3
HYBRID_LOW_BANDS = 16 HYBRID_LOW_BANDS = 16
ANALYSIS_SYNTHESIS_LATENCY_SAMPLES = 448 ANALYSIS_SYNTHESIS_LATENCY_SAMPLES = 448
# The captured renderer's hybrid band order: destination[k] = low[HYBRID_ORDER[k]]. # Hybrid band order: destination[k] = low[HYBRID_ORDER[k]].
HYBRID_ORDER = (0, 7, 1, 6, 2, 9, 3, 4, 10, 5, 11, 15, 8, 12, 13, 14) HYBRID_ORDER = (0, 7, 1, 6, 2, 9, 3, 4, 10, 5, 11, 15, 8, 12, 13, 14)
# Bands 1 and 3 are conjugated after the permutation. # Bands 1 and 3 are conjugated after the permutation.
@@ -65,7 +57,7 @@ def _load_tables(path_string: str) -> dict[str, np.ndarray]:
def load_kernel_tables(path: str | Path = DEFAULT_CAPTURED_KERNEL_DATA def load_kernel_tables(path: str | Path = DEFAULT_CAPTURED_KERNEL_DATA
) -> dict[str, np.ndarray]: ) -> dict[str, np.ndarray]:
"""Load and cache the captured Rosella kernel tables.""" """Load and cache the 11-tap Rosella bank's kernel tables."""
return _load_tables(str(Path(path).expanduser().resolve())) return _load_tables(str(Path(path).expanduser().resolve()))
+6 -2
View File
@@ -11,14 +11,18 @@ from speaker_renderer import PythonSpeakerRenderer, payload_events_from_index, r
FRAME_SAMPLES = 1536 FRAME_SAMPLES = 1536
def create_speaker_renderer(layout: str | SpeakerLayout, bed_channels=None): def create_speaker_renderer(layout: str | SpeakerLayout, bed_channels=None,
trim_meta=None, *, parameters=None):
"""Create a stateful speaker renderer and return ``(renderer, info)``.""" """Create a stateful speaker renderer and return ``(renderer, info)``."""
target = get_speaker_layout(layout) if isinstance(layout, str) else layout target = get_speaker_layout(layout) if isinstance(layout, str) else layout
renderer = PythonSpeakerRenderer(target, bed_channels=bed_channels) renderer = PythonSpeakerRenderer(target, bed_channels=bed_channels,
trim_meta=trim_meta, parameters=parameters)
return renderer, { return renderer, {
"name": "python", "name": "python",
"layout": target.name, "layout": target.name,
"bed_objects": len(getattr(renderer, "bed_targets", {}) or {}), "bed_objects": len(getattr(renderer, "bed_targets", {}) or {}),
"initial_warp_mode": renderer.parameters.warp_mode,
"initial_trim_mode": renderer.parameters.trim_mode,
} }
+33 -4
View File
@@ -128,11 +128,11 @@ _RAW_LAYOUTS = {'20': {'out_ch_config': 0,
'axis0_groups': ((0, 1), (2, 3)), 'axis0_groups': ((0, 1), (2, 3)),
'axis1_groups': (), 'axis1_groups': (),
'mode': 2}, 'mode': 2},
{'coordinates_q15': ((0, 32767, 0), (32767, 32767, 0)), {'coordinates_q15': ((0, 0, 0), (32767, 0, 0), (0, 32767, 0), (32767, 32767, 0)),
'speaker_ids': (2, 3), 'speaker_ids': (0, 1, 2, 3),
'axis0_groups': ((0, 1),), 'axis0_groups': ((0, 1), (2, 3)),
'axis1_groups': (), 'axis1_groups': (),
'mode': 1}, 'mode': 2},
{'coordinates_q15': ((0, 0, 0), (32767, 0, 0)), {'coordinates_q15': ((0, 0, 0), (32767, 0, 0)),
'speaker_ids': (0, 1), 'speaker_ids': (0, 1),
'axis0_groups': ((0, 1),), 'axis0_groups': ((0, 1),),
@@ -774,6 +774,34 @@ def _make_layout(name: str, item: dict) -> SpeakerLayout:
SPEAKER_LAYOUTS = {name: _make_layout(name, item) for name, item in _RAW_LAYOUTS.items()} SPEAKER_LAYOUTS = {name: _make_layout(name, item) for name, item in _RAW_LAYOUTS.items()}
def _make_six_zero():
"""6.0: L/R, side pair and rear pair without centre or LFE."""
base = SPEAKER_LAYOUTS["71"]
channels = ("L", "R", "Ls", "Rs", "Lrs", "Rrs")
source_ids = (0, 1, 4, 5, 6, 7)
remap = {old: new for new, old in enumerate(source_ids)}
regions = []
for region in base.regions:
kept = [index for index, speaker in enumerate(region.speaker_ids)
if speaker in remap]
if not kept:
continue
local = {old: new for new, old in enumerate(kept)}
def groups(items):
return tuple(tuple(local[index] for index in group if index in local)
for group in items if any(index in local for index in group))
regions.append(RegionGeometry(
tuple(region.coordinates_q15[index] for index in kept),
tuple(remap[region.speaker_ids[index]] for index in kept),
groups(region.axis0_groups), groups(region.axis1_groups),
region.mode))
return SpeakerLayout("60", 255, 25, channels,
tuple(range(6)), tuple(regions))
SPEAKER_LAYOUTS["60"] = _make_six_zero()
SPEAKER_LAYOUT_ALIASES = { SPEAKER_LAYOUT_ALIASES = {
"2.0": "20", "2.0": "20",
"3.1": "31", "3.1": "31",
@@ -786,6 +814,7 @@ SPEAKER_LAYOUT_ALIASES = {
"7.1.4": "714", "7.1.4": "714",
"9.1.4": "914", "9.1.4": "914",
"9.1.6": "916", "9.1.6": "916",
"6.0": "60",
} }
SPEAKER_LAYOUT_CHOICES = tuple(SPEAKER_LAYOUT_ALIASES) SPEAKER_LAYOUT_CHOICES = tuple(SPEAKER_LAYOUT_ALIASES)
SPEAKER_LAYOUT_DISPLAY_NAMES = {value: key for key, value in SPEAKER_LAYOUT_ALIASES.items()} SPEAKER_LAYOUT_DISPLAY_NAMES = {value: key for key, value in SPEAKER_LAYOUT_ALIASES.items()}
+609 -47
View File
@@ -7,25 +7,82 @@ writes a lower-precision output format.
from __future__ import annotations from __future__ import annotations
from collections import defaultdict from collections import defaultdict
from functools import lru_cache
import math import math
from typing import Iterable from typing import Iterable
import numpy as np import numpy as np
from oamd_bits import JocFieldState, frame_update import oamd_math
from oamd_bits import JocFieldState, frame_update, parse_trim
from oamd_params import metadata_params
from variant_error import UnsupportedVariantError
from speaker_layouts import RegionGeometry, SpeakerLayout, get_speaker_layout from speaker_layouts import RegionGeometry, SpeakerLayout, get_speaker_layout
from trim_balance import (SpeakerRenderParameters, apply_balance,
explicit_position_gain, parameters_from_oamd)
Q15_SCALE = 32768.0 Q15_SCALE = 32768.0
Q15_MAX = 32767.0 / Q15_SCALE Q15_MAX = 32767.0 / Q15_SCALE
GAIN_SNAP_THRESHOLD = 1.0e-4 GAIN_SNAP_THRESHOLD = 1.0e-4
ZONE_TO_REGION = (0, 1, 2, 3, 4, 5, 6, 0)
LFE_BED_LABELS = ("RC_LFE", "RC_LFE2")
BED_LABEL_INDEX = {label: index + 1 for index, label in
enumerate(("RC_L", "RC_R", "RC_C", "RC_LFE",
"RC_LS", "RC_RS", "RC_LB", "RC_RB",
"RC_TFL", "RC_TFR", "RC_TSL", "RC_TSR",
"RC_TBL", "RC_TBR", "RC_LW", "RC_RW",
"RC_LFE2"))}
def _stable_norm(vector):
"""Reference vector normalization: tiny-norm floor and unit clamp."""
values = np.asarray(vector, dtype=np.float64)
total = float(np.dot(values, values))
if total == 0.0:
return np.zeros_like(values)
result = values / max(math.sqrt(total), 1.0e-5)
return np.minimum(result, 1.0)
def _region_vector(layout, region_index, u, floor_v, top_v, w,
enable_height):
"""Uncompensated point-source vector in ``layout`` internal order."""
region = layout.regions[int(region_index)]
coordinates = _coordinates(region)
floor = _plane_gains(region, coordinates, region.axis0_groups, u,
floor_v, region.mode)
gains = floor
if region.mode == 3:
top = _plane_gains(region, coordinates, region.axis1_groups, u,
top_v, 3)
height = min(max(float(w) if enable_height else 0.0, 0.0),
Q15_MAX)
if height >= Q15_MAX:
gains = top
elif height > 0.0:
floor_weight, height_weight = equal_power_pair(height)
gains = floor * floor_weight + top * height_weight
vector = np.zeros(layout.channel_count, dtype=np.float64)
for value, speaker in zip(gains, region.speaker_ids):
vector[speaker] = value
return vector
def _fold_vector(calc, final, warp_mode, vector):
"""Apply the existing warp 2/3 surround fold to a calculation vector."""
if calc is final:
return vector
return (_surround_fold_matrix(calc.speaker_bitfield,
final.speaker_bitfield, warp_mode)
@ vector)
def bed_object_targets(bed_channels) -> dict[int, tuple[float, float, float]]: def bed_object_targets(bed_channels) -> dict[int, tuple[float, float, float]]:
"""JOC 对象槽 → 固定方位 ``(u, v, w)``;只包含 bed 声道。 """JOC 对象槽 → 固定方位 ``(u, v, w)``;只包含 bed 声道。
bed 声道的方位由声道标签定义(ETSI TS 103 420 Table B.10),与目标扬声器布局 bed 声道的方位由声道标签定义,与目标扬声器布局无关:同一个声道无论输出哪种
无关:同一个声道无论输出哪种布局都落在同一个源位置,再交给与动态对象相同的 布局都落在同一个源位置,再交给与动态对象相同的点声源渲染路径。LFE 不占对象
点声源渲染路径。LFE 不占对象槽,单独处理。 槽,单独处理。
""" """
from bed_layout import adm_xyz_to_render_coordinates, bed_object_positions from bed_layout import adm_xyz_to_render_coordinates, bed_object_positions
@@ -34,8 +91,9 @@ def bed_object_targets(bed_channels) -> dict[int, tuple[float, float, float]]:
def _all_object_targets(layout: SpeakerLayout, state: JocFieldState, def _all_object_targets(layout: SpeakerLayout, state: JocFieldState,
bed_targets=None) -> np.ndarray: bed_targets=None, parameters=None) -> np.ndarray:
bed_targets = bed_targets or {} bed_targets = bed_targets or {}
parameters = parameters or parameters_from_oamd(None, layout)
result = np.zeros((15, layout.channel_count), dtype=np.float64) result = np.zeros((15, layout.channel_count), dtype=np.float64)
for object_index in range(1, 16): for object_index in range(1, 16):
fixed = bed_targets.get(object_index) fixed = bed_targets.get(object_index)
@@ -46,7 +104,8 @@ def _all_object_targets(layout: SpeakerLayout, state: JocFieldState,
v = state.q[(object_index, "q2")] / Q15_SCALE v = state.q[(object_index, "q2")] / Q15_SCALE
w = state.q[(object_index, "q3")] / Q15_SCALE w = state.q[(object_index, "q3")] / Q15_SCALE
result[object_index - 1] = render_point_gains( result[object_index - 1] = render_point_gains(
layout, u, v, w, standard_order=False) layout, u, v, w, standard_order=False, parameters=parameters,
bypass_trim=parameters.bypasses(object_index))
return result return result
@@ -172,16 +231,102 @@ def _plane_gains(region: RegionGeometry, coordinates: np.ndarray, groups,
def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float, def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float,
*, region_index: int = 0, enable_height: bool = True, *, region_index: int = 0, enable_height: bool = True,
object_gain: float = 1.0, standard_order: bool = True) -> np.ndarray: object_gain: float = 1.0, standard_order: bool = True,
parameters: SpeakerRenderParameters | None = None,
bypass_trim: bool = False) -> np.ndarray:
"""Render one point object to a target speaker layout using float64.""" """Render one point object to a target speaker layout using float64."""
target = get_speaker_layout(layout) if isinstance(layout, str) else layout target = get_speaker_layout(layout) if isinstance(layout, str) else layout
controls = parameters or parameters_from_oamd(None, target)
source = _surround_fold_layout(target) if controls.warp_mode in (2, 3) else target
# Geometry may expand to 7.1.x; calibration always uses the final layout.
internal = _render_point_gains(source, u, v, w, region_index=region_index,
enable_height=enable_height, object_gain=object_gain,
parameters=controls, bypass_trim=bypass_trim,
compensation_bitfield=target.speaker_bitfield)
if source is not target:
internal = _surround_fold_matrix(source.speaker_bitfield, target.speaker_bitfield,
controls.warp_mode) @ internal
if standard_order:
return internal[list(target.internal_to_standard)]
return internal
@lru_cache(maxsize=16)
def _surround_fold_layout(target: SpeakerLayout) -> SpeakerLayout:
"""Add a rear pair for one-pair surround layouts, keeping the height layer."""
if not floor_y_exponent(target.speaker_bitfield):
return target
expanded = {"51": "71", "512": "712", "514": "714"}
if target.name in expanded:
return get_speaker_layout(expanded[target.name])
if target.name == "60":
return target
if target.name != "40":
raise ValueError(f"unsupported surround fold layout: {target.name}")
# 4.0 expands to six full-band speakers without adding centre or LFE.
base = get_speaker_layout("71")
regions = []
speaker_ids = {0: 0, 1: 1, 4: 2, 5: 3, 6: 4, 7: 5}
for region in base.regions:
kept = [i for i, speaker in enumerate(region.speaker_ids) if speaker in speaker_ids]
indices = {old: new for new, old in enumerate(kept)}
def groups(items):
return tuple(tuple(indices[i] for i in group if i in indices)
for group in items if any(i in indices for i in group))
regions.append(RegionGeometry(
tuple(region.coordinates_q15[i] for i in kept),
tuple(speaker_ids[region.speaker_ids[i]] for i in kept),
groups(region.axis0_groups), groups(region.axis1_groups), region.mode))
return SpeakerLayout("60", 255, target.speaker_bitfield | 0x10,
("L", "R", "Ls", "Rs", "Lrs", "Rrs"),
tuple(range(6)), tuple(regions))
@lru_cache(maxsize=32)
def _surround_fold_matrix(source_bitfield: int, target_bitfield: int,
warp_mode: int) -> np.ndarray:
"""Copy shared speakers and fold the rear pair into the target surround pair."""
def speakers(bitfield):
mask = expand_speaker_bitfield(bitfield)
return tuple(i for i in range(mask.bit_length()) if mask & (1 << i))
source, target = speakers(source_bitfield), speakers(target_bitfield)
matrix = np.zeros((len(target), len(source)), dtype=np.float64)
same_side, cross_side = (math.sqrt(3.0) / 2.0, 0.5) if warp_mode == 2 else (1.0, 0.0)
# Speaker IDs 4/5 are side surrounds; 6/7 are the rear pair.
for column, speaker in enumerate(source):
if speaker in target:
matrix[target.index(speaker), column] = 1.0
elif speaker in (6, 7):
matrix[target.index(speaker - 2), column] = same_side
matrix[target.index(5 if speaker == 6 else 4), column] = cross_side
matrix.flags.writeable = False
return matrix
def _render_point_gains(target: SpeakerLayout, u: float, v: float, w: float,
*, region_index: int, enable_height: bool, object_gain: float,
parameters: SpeakerRenderParameters, bypass_trim: bool,
compensation_bitfield: int) -> np.ndarray:
"""Evaluate point gains before any surround fold or output-channel reorder."""
controls = parameters
region = target.regions[int(region_index)] region = target.regions[int(region_index)]
coordinates = _coordinates(region) coordinates = _coordinates(region)
floor_v = min(max(math.ldexp(float(v), floor_y_exponent(target.speaker_bitfield)), 0.0), 1.0) v, w = float(v), max(float(w), 0.0)
floor_v = min(max(math.ldexp(v, floor_y_exponent(
target.speaker_bitfield, controls.warp_mode != 0)), 0.0), 1.0)
top_v = min(max(v, 0.0), 1.0)
trim_v = v
if controls.trim_mode == "explicit" and not bypass_trim:
# Apply warp -> balance for floor, balance(original Y) for
# top and trim. These coordinates must not be collapsed into one Y.
listener, overhead = controls.listener_balance, controls.overhead_balance
floor_v = apply_balance(floor_v, w, listener, overhead)
top_v = apply_balance(top_v, w, listener, overhead)
trim_v = apply_balance(v, w, listener, overhead)
floor = _plane_gains(region, coordinates, region.axis0_groups, u, floor_v, region.mode) floor = _plane_gains(region, coordinates, region.axis0_groups, u, floor_v, region.mode)
point_gains = floor point_gains = floor
if region.mode == 3: if region.mode == 3:
top = _plane_gains(region, coordinates, region.axis1_groups, u, float(v), 3) top = _plane_gains(region, coordinates, region.axis1_groups, u, top_v, 3)
height = min(max(float(w) if enable_height else 0.0, 0.0), Q15_MAX) height = min(max(float(w) if enable_height else 0.0, 0.0), Q15_MAX)
if height >= Q15_MAX: if height >= Q15_MAX:
point_gains = top point_gains = top
@@ -189,11 +334,14 @@ def render_point_gains(layout: str | SpeakerLayout, u: float, v: float, w: float
floor_weight, height_weight = equal_power_pair(height) floor_weight, height_weight = equal_power_pair(height)
point_gains = floor * floor_weight + top * height_weight point_gains = floor * floor_weight + top * height_weight
internal = np.zeros(target.channel_count, dtype=np.float64) internal = np.zeros(target.channel_count, dtype=np.float64)
gain = position_gain(target.speaker_bitfield, v, w) * float(object_gain) gain = float(object_gain)
if not bypass_trim:
if controls.trim_mode == "automatic":
gain *= position_gain(compensation_bitfield, trim_v, w)
elif controls.trim_mode == "explicit":
gain *= explicit_position_gain(controls, u, trim_v, w)
for point_gain, speaker_id in zip(point_gains, region.speaker_ids): for point_gain, speaker_id in zip(point_gains, region.speaker_ids):
internal[speaker_id] = point_gain * gain internal[speaker_id] = point_gain * gain
if standard_order:
return internal[list(target.internal_to_standard)]
return internal return internal
@@ -207,18 +355,36 @@ def ramp_block_count(duration_samples: int, block_size: int = 32,
class PythonSpeakerRenderer: class PythonSpeakerRenderer:
"""Stateful float64 speaker renderer.""" """Stateful float64 speaker renderer.
Two input shapes are accepted. Payloads that expand to block-level object
state (``frame_update`` with ``object_blocks``) use the metadata path:
object gain/inactive, region/elevation, size/snap/divergence/refinement,
priority budget, ISF and bed distribution. Older payload dictionaries keep
the previous point-position path so existing callers and tests still work.
"""
def __init__(self, layout: str | SpeakerLayout, block_size: int = 32, def __init__(self, layout: str | SpeakerLayout, block_size: int = 32,
bed_channels=None): bed_channels=None, trim_meta=None, *, parameters=None,
size_budget: int = 0):
self.layout = get_speaker_layout(layout) if isinstance(layout, str) else layout self.layout = get_speaker_layout(layout) if isinstance(layout, str) else layout
self.block_size = int(block_size) self.block_size = int(block_size)
if self.block_size <= 0: if self.block_size <= 0:
raise ValueError("block_size must be positive") raise ValueError("block_size must be positive")
self.bed_targets = bed_object_targets(bed_channels) self.bed_targets = bed_object_targets(bed_channels)
if parameters is not None and trim_meta is not None:
raise ValueError("pass normalized parameters or OAMD trim_meta, not both")
self._parameters_override = parameters
self._initial_parameters = parameters or parameters_from_oamd(trim_meta, self.layout)
self._size_budget = int(size_budget)
if self._size_budget < 0:
raise ValueError("size_budget must be non-negative")
self._raw_trim_mode = (trim_meta or {}).get("global_trim_mode", 0)
self._meta_params = metadata_params()
self.reset() self.reset()
def reset(self): def reset(self):
self.parameters = self._initial_parameters
channels = self.layout.channel_count channels = self.layout.channel_count
self._state = JocFieldState() self._state = JocFieldState()
self._current = np.zeros((15, channels), dtype=np.float64) self._current = np.zeros((15, channels), dtype=np.float64)
@@ -226,11 +392,341 @@ class PythonSpeakerRenderer:
self._step = np.zeros_like(self._current) self._step = np.zeros_like(self._current)
self._remaining = np.zeros((15, channels), dtype=np.int32) self._remaining = np.zeros((15, channels), dtype=np.int32)
self._window = np.arange(self.block_size, dtype=np.float64) / float(self.block_size) self._window = np.arange(self.block_size, dtype=np.float64) / float(self.block_size)
self._meta_mode = False
self._meta_ready = False
self._n_meta = 0
self._meta_slots: list[int | None] = []
self._meta_labels: list[str | None] = []
self._mc = None
self._mt = None
self._ms = None
self._mr = None
def _apply_payload(self, payload): # -------------------------------------------------------------- metadata state
update = frame_update(payload) def _update_trim(self, payload, update=None):
trim = update.get("trim") if update is not None else None
if trim is None and update is None:
trim = parse_trim(payload)
if trim is not None and self._parameters_override is None:
self.parameters = parameters_from_oamd(trim, self.layout)
if isinstance(trim, dict) and "global_trim_mode" in trim:
self._raw_trim_mode = trim["global_trim_mode"]
def _init_meta(self, update):
self._isf_cache = {}
program = update["program"]
kinds = update["kinds"]
count = int(update["object_count"])
labels = [label for group in program["bed_channels"] for label in group]
num_bed = int(program["num_bed_objects"])
if len(labels) != num_bed:
if program["dynamic_object_only"]:
labels = ["RC_LFE"] if program["lfe_present"] else []
if len(labels) != num_bed:
raise UnsupportedVariantError(
"speaker", "bed_declaration_mismatch",
"OAMD bed 声明数量与节目分区不一致",
details={"declared": len(labels), "num_bed_objects": num_bed})
slots = []
slot = 1
for object_index, kind in enumerate(kinds):
label = labels[object_index] if object_index < len(labels) else None
if kind == "bed" and label in LFE_BED_LABELS:
slots.append(0)
else:
if slot > 15:
raise UnsupportedVariantError(
"speaker", "object_slot_overflow",
"元数据对象数量超出 15 个 JOC 对象槽",
details={"object_count": count})
slots.append(slot)
slot += 1
if not program["dynamic_object_only"]:
declared = program["num_dynamic_objects"]
if (declared is not None
and num_bed + int(program["num_isf_objects"]) + int(declared)
!= count):
raise UnsupportedVariantError(
"speaker", "program_object_count_mismatch",
"OAMD 节目声明的对象分区与对象数不一致",
details={"object_count": count,
"num_bed_objects": num_bed,
"num_isf_objects": program["num_isf_objects"],
"num_dynamic_objects": declared})
if program["num_isf_objects"]:
isf_idx = program["isf_idx"]
config = (self._meta_params["isf"]["configs"][isf_idx]
if isf_idx is not None and isf_idx < 6 else None)
if config is None or config["counts"][4] != program["num_isf_objects"]:
raise UnsupportedVariantError(
"speaker", "isf_configuration_mismatch",
"ISF 配置数量与节目声明不一致",
details={"isf_idx": isf_idx,
"num_isf_objects": program["num_isf_objects"]})
if count != self._n_meta or slots != self._meta_slots:
channels = self.layout.channel_count
self._n_meta = count
self._meta_slots = slots
self._mc = np.zeros((count, channels), dtype=np.float64)
self._mt = np.zeros((count, channels), dtype=np.float64)
self._ms = np.zeros((count, channels), dtype=np.float64)
self._mr = np.zeros((count, channels), dtype=np.int32)
self._meta_labels = [labels[object_index]
if object_index < len(labels) else None
for object_index in range(count)]
self._meta_mode = True
def _block_targets(self, update, block_index):
count = int(update["object_count"])
channels = self.layout.channel_count
targets = np.zeros((count, channels), dtype=np.float64)
blocks = update["object_blocks"]
kinds = update["kinds"]
zero_size = set()
size_objects = []
for object_index in range(count):
state = blocks[object_index][block_index]
if (kinds[object_index] == "dynamic" and not state["not_active"]
and state["size_mode"] in (1, 2)
and not state["snap"] and state["divergence_q"] == 0):
size_objects.append(object_index)
if 0 < self._size_budget < len(size_objects):
ranked = sorted(
size_objects,
key=lambda index: (-((int(blocks[index][block_index]["priority_q"])
+ 512) >> 10), index))
zero_size = set(ranked[self._size_budget:])
for object_index in range(count):
state = blocks[object_index][block_index]
if state["not_active"]:
continue
spatial = self._object_spatial(update, object_index, block_index,
object_index in zero_size)
gain = float(state["gain_q"]) / 4096.0
targets[object_index] = spatial * gain
return targets
def _object_spatial(self, update, object_index, block_index, force_zero_size):
kind = update["kinds"][object_index]
if kind == "dynamic":
return self._dynamic_spatial(update, object_index, block_index,
force_zero_size)
if kind == "bed":
return self._bed_spatial(update, object_index)
if kind == "isf":
return self._isf_spatial(update, object_index)
return np.zeros(self.layout.channel_count, dtype=np.float64)
# -------------------------------------------------------------- dynamic path
def _dynamic_spatial(self, update, object_index, block_index,
force_zero_size):
state = update["object_blocks"][object_index][block_index]
final = self.layout
warp = self.parameters.warp_mode
calc = _surround_fold_layout(final) if warp in (2, 3) else final
region_index = ZONE_TO_REGION[int(state["zone_idx"]) & 7]
u = float(state["qx"]) / Q15_SCALE
v = float(state["qy"]) / Q15_SCALE
w = max(float(state["qz"]) / Q15_SCALE, 0.0)
enable_height = bool(state["enable_elevation"])
if not enable_height:
w = 0.0
bypass = self.parameters.bypasses(object_index)
floor_v = min(max(math.ldexp(v, floor_y_exponent(
calc.speaker_bitfield, warp != 0)), 0.0), 1.0)
top_v = min(max(v, 0.0), 1.0)
trim_v = v
if self.parameters.trim_mode == "explicit" and not bypass:
listener = self.parameters.listener_balance
overhead = self.parameters.overhead_balance
floor_v = apply_balance(floor_v, w, listener, overhead)
top_v = apply_balance(top_v, w, listener, overhead)
trim_v = apply_balance(v, w, listener, overhead)
region = calc.regions[region_index]
coordinates = _coordinates(region)
if state["snap"]:
point = _region_vector(calc, region_index, u, floor_v, top_v, w,
enable_height)
best = -1.0
chosen = 0
for index, speaker in enumerate(region.speaker_ids):
value = float(point[speaker])
if value > best:
best = value
chosen = index
speaker_u, speaker_v, speaker_w = coordinates[chosen]
scale = 1.0
distance = ((u - speaker_u) ** 2
+ (min(v, scale) - scale * speaker_v) ** 2
+ (w - speaker_w) ** 2)
if distance < 0.04:
vector = np.zeros(calc.channel_count, dtype=np.float64)
vector[region.speaker_ids[chosen]] = 1.0
comp = (float(speaker_u), float(speaker_v), float(speaker_w))
else:
vector = point
comp = (u, trim_v, w)
elif state["divergence_q"] > 0:
spread = min(float(state["divergence_q"]) / Q15_SCALE, u, 1.0 - u)
left = _region_vector(calc, region_index, u - spread, floor_v,
top_v, w, enable_height)
right = _region_vector(calc, region_index, u + spread, floor_v,
top_v, w, enable_height)
vector = _stable_norm(left + right)
comp = (u, trim_v, w)
elif state["size_mode"] in (1, 2) and not force_zero_size:
size_q = state["size_q"]
if size_q[0] == size_q[1] == size_q[2]:
q_size = int(size_q[0])
else:
q_size = (int(size_q[0]) + int(size_q[1]) + int(size_q[2])) // 3
size = q_size / Q15_SCALE
t = oamd_math.size_curve_value(size)
point = _region_vector(calc, region_index, u, floor_v, top_v, w,
enable_height)
if t < 0.000025:
vector = point
else:
query = (u, (1.0 - w) * floor_v + w * top_v, w)
spread = np.asarray(oamd_math.size_targets(
self._meta_params, calc, region_index, query, size,
height=enable_height), dtype=np.float64)
if t > 0.05:
vector = spread
else:
angle = 20.0 * t
if angle >= 0.99996948:
cosine, sine = 0.0, 1.0
else:
cosine = math.cos(angle * math.pi / 2.0)
sine = math.sin(angle * math.pi / 2.0)
vector = _stable_norm(0.5 * cosine * point
+ 0.5 * sine * spread)
comp = (u, trim_v, w)
else:
vector = _region_vector(calc, region_index, u, floor_v, top_v, w,
enable_height)
comp = (u, trim_v, w)
compensation = 1.0
if not bypass:
if self.parameters.trim_mode == "automatic":
compensation = position_gain(final.speaker_bitfield,
comp[1], comp[2])
elif self.parameters.trim_mode == "explicit":
compensation = explicit_position_gain(self.parameters,
comp[0], comp[1], comp[2])
return _fold_vector(calc, final, warp, vector) * compensation
# -------------------------------------------------------------- bed path
def _bed_spatial(self, update, object_index):
final = self.layout
warp = self.parameters.warp_mode
calc = _surround_fold_layout(final) if warp in (2, 3) else final
label = self._meta_labels[object_index]
bed_index = BED_LABEL_INDEX.get(label)
if bed_index is None:
return np.zeros(final.channel_count, dtype=np.float64)
program = update["program"]
route, weights = oamd_math.bed_distribution(
calc.name, bed_index, int(program["bed_mask"]),
bool(program["b_bed_chan_distribute"]))
bypass = self.parameters.bypasses(object_index)
mode = {"automatic": 0, "disabled": 1, "explicit": 2}[
self.parameters.trim_mode]
if route != -1:
position = oamd_math.bed_reference_position(calc.name, route)
qx, qy, qz = (int(value) for value in position)
if qz > 0:
qx = 32767 if qx > 16384 else 0
u, v = qx / Q15_SCALE, qy / Q15_SCALE
w = max(qz, 0) / Q15_SCALE
floor_v = min(max(math.ldexp(v, floor_y_exponent(
calc.speaker_bitfield, warp != 0)), 0.0), 1.0)
top_v = min(max(v, 0.0), 1.0)
vector = _region_vector(calc, 0, u, floor_v, top_v, w, True)
compensation = 1.0
if not bypass and self.parameters.trim_mode != "disabled":
if self.parameters.trim_mode == "automatic":
compensation = position_gain(final.speaker_bitfield, v, w)
else:
compensation = explicit_position_gain(
self.parameters, u, v, w)
else:
vector = np.asarray(weights, dtype=np.float64).copy()
levels = (self.parameters.surround_db,
self.parameters.height_db,
self.parameters.centre_db)
compensation = float(oamd_math.bed_compensation(
final.name, bed_index, mode, bypass, levels))
return _fold_vector(calc, final, warp, vector) * compensation
# -------------------------------------------------------------- ISF path
def _isf_spatial(self, update, object_index):
program = update["program"]
isf_idx = int(program["isf_idx"])
row = object_index - int(program["num_bed_objects"])
final = self.layout
warp = self.parameters.warp_mode
calc = _surround_fold_layout(final) if warp in (2, 3) else final
key = (calc.name, isf_idx)
matrix = self._isf_cache.get(key)
if matrix is None:
matrix = oamd_math.isf_matrix(self._meta_params, calc.name, isf_idx)
self._isf_cache[key] = matrix
vector = np.asarray(matrix[row], dtype=np.float64)
disabled = (int(self._raw_trim_mode) == 1
or self.parameters.bypasses(object_index))
compensation = float(oamd_math.isf_compensation(
self._meta_params, final.name, isf_idx, row, disabled))
return _fold_vector(calc, final, warp, vector) * compensation
# -------------------------------------------------------------- scheduling
def _apply_meta_targets(self, targets, ramp_samples):
channels = self.layout.channel_count
if self._mc is None or self._mc.shape != (len(targets), channels):
self._mc = np.zeros((len(targets), channels), dtype=np.float64)
self._mt = np.zeros_like(self._mc)
self._ms = np.zeros_like(self._mc)
self._mr = np.zeros((len(targets), channels), dtype=np.int32)
difference = np.asarray(targets, dtype=np.float64) - self._mc
significant = np.abs(difference) >= GAIN_SNAP_THRESHOLD
self._mt[...] = targets
self._mc[~significant] = self._mt[~significant]
self._ms[~significant] = 0.0
self._mr[~significant] = 0
blocks = ramp_block_count(int(ramp_samples), self.block_size)
if blocks:
self._ms[significant] = difference[significant] / float(blocks)
self._mr[significant] = blocks
else:
self._mc[significant] = self._mt[significant]
self._ms[significant] = 0.0
self._mr[significant] = 0
self._meta_ready = True
def _schedule_payload(self, update, event_sample):
if "object_blocks" not in update or not update["object_count"]:
return False
if update.get("trim") is not None:
self._update_trim(None, update)
self._init_meta(update)
for block_index, block in enumerate(update["blocks"]):
sample = (int(event_sample) + int(update["sample_offset"])
+ int(block["block_offset_samples"]))
aligned = align_metadata_sample(sample, self.block_size)
targets = self._block_targets(update, block_index)
self._pending.setdefault(aligned // self.block_size, []).append(
(targets, block["ramp_duration_samples"]))
return True
# -------------------------------------------------------------- legacy path
def _apply_legacy(self, update):
trim = parse_trim(self._current_payload)
if trim is not None and self._parameters_override is None:
self.parameters = parameters_from_oamd(trim, self.layout)
self._state.apply(update["values"]) self._state.apply(update["values"])
new_target = _all_object_targets(self.layout, self._state, self.bed_targets) new_target = _all_object_targets(self.layout, self._state,
self.bed_targets, self.parameters)
blocks = ramp_block_count(update["ramp_duration_samples"], self.block_size) blocks = ramp_block_count(update["ramp_duration_samples"], self.block_size)
difference = new_target - self._current difference = new_target - self._current
significant = np.abs(difference) >= GAIN_SNAP_THRESHOLD significant = np.abs(difference) >= GAIN_SNAP_THRESHOLD
@@ -246,6 +742,62 @@ class PythonSpeakerRenderer:
self._step[significant] = 0.0 self._step[significant] = 0.0
self._remaining[significant] = 0 self._remaining[significant] = 0
def _apply_payload(self, payload):
self._current_payload = payload
update = frame_update(payload)
if "object_blocks" in update and update["object_count"]:
self._update_trim(payload, update)
self._init_meta(update)
targets = self._block_targets(update, 0)
self._apply_meta_targets(targets,
update["blocks"][0]["ramp_duration_samples"])
else:
self._apply_legacy(update)
# -------------------------------------------------------------- process
def _render_meta_block(self, source, start, stop, out_block):
for object_index in range(self._n_meta):
slot = self._meta_slots[object_index]
if slot is None:
continue
active = self._mr[object_index] > 0
gains = np.broadcast_to(self._mt[object_index],
(self.block_size, self.layout.channel_count)).copy()
if np.any(active):
gains[:, active] = (
self._mc[object_index, active][None, :]
+ self._window[:, None] * self._ms[object_index, active][None, :]
)
out_block += source[start:stop, slot, None] * gains
if np.any(active):
self._mc[object_index, active] += self._ms[object_index, active]
self._mr[object_index, active] -= 1
finished = self._mr[object_index] == 0
self._mc[object_index, finished] = self._mt[object_index, finished]
def _render_legacy_block(self, source, start, stop, out_block):
channels = self.layout.channel_count
if "LFE" in self.layout.channels:
out_block[:, 3] = source[start:stop, 0].astype(np.float64, copy=False)
for object_index in range(15):
active = self._remaining[object_index] > 0
gains = np.broadcast_to(self._target[object_index],
(self.block_size, channels)).copy()
if np.any(active):
gains[:, active] = (
self._current[object_index, active][None, :]
+ self._window[:, None] * self._step[object_index, active][None, :]
)
out_block += (
source[start:stop, object_index + 1, None].astype(np.float64, copy=False)
* gains
)
if np.any(active):
self._current[object_index, active] += self._step[object_index, active]
self._remaining[object_index, active] -= 1
finished = self._remaining[object_index] == 0
self._current[object_index, finished] = self._target[object_index, finished]
def process(self, objects16: np.ndarray, events=(), *, standard_order: bool = True, def process(self, objects16: np.ndarray, events=(), *, standard_order: bool = True,
output: np.ndarray | None = None) -> np.ndarray: output: np.ndarray | None = None) -> np.ndarray:
source = np.asarray(objects16) source = np.asarray(objects16)
@@ -261,44 +813,48 @@ class PythonSpeakerRenderer:
raise ValueError((output.shape, output.dtype)) raise ValueError((output.shape, output.dtype))
else: else:
output[...] = 0.0 output[...] = 0.0
if "LFE" in self.layout.channels:
output[:, 3] = source[:, 0].astype(np.float64, copy=False)
events_by_block: dict[int, list[bytes]] = defaultdict(list) events_by_block: dict[int, list[tuple[int, bytes]]] = defaultdict(list)
for sample_offset, payload in events: for sample_offset, payload in events:
if not 0 <= int(sample_offset) <= total_samples: if not 0 <= int(sample_offset) <= total_samples:
raise ValueError(f"metadata offset outside process call: {sample_offset}") raise ValueError(f"metadata offset outside process call: {sample_offset}")
block = align_metadata_sample(sample_offset, self.block_size) // self.block_size block = align_metadata_sample(sample_offset, self.block_size) // self.block_size
events_by_block[block].append(bytes(payload)) events_by_block[block].append((align_metadata_sample(
sample_offset, self.block_size), bytes(payload)))
self._pending: dict[int, list] = {}
total_blocks = total_samples // self.block_size total_blocks = total_samples // self.block_size
for block in range(total_blocks): for block in range(total_blocks):
for payload in events_by_block.get(block, ()): for event_sample, payload in events_by_block.get(block, ()):
self._apply_payload(payload) update = frame_update(payload)
if "object_blocks" in update and update["object_count"]:
self._schedule_payload(update, event_sample)
else:
self._current_payload = payload
self._apply_legacy(update)
for targets, ramp_samples in self._pending.pop(block, ()):
self._apply_meta_targets(targets, ramp_samples)
start = block * self.block_size start = block * self.block_size
stop = start + self.block_size stop = start + self.block_size
out_block = output[start:stop] out_block = output[start:stop]
for object_index in range(15): if self._meta_mode and self._meta_ready:
active = self._remaining[object_index] > 0 self._render_meta_block(source, start, stop, out_block)
gains = np.broadcast_to(self._target[object_index], else:
(self.block_size, channels)).copy() self._render_legacy_block(source, start, stop, out_block)
if np.any(active):
gains[:, active] = (
self._current[object_index, active][None, :]
+ self._window[:, None] * self._step[object_index, active][None, :]
)
out_block += (
source[start:stop, object_index + 1, None].astype(np.float64, copy=False)
* gains
)
if np.any(active):
self._current[object_index, active] += self._step[object_index, active]
self._remaining[object_index, active] -= 1
finished = self._remaining[object_index] == 0
self._current[object_index, finished] = self._target[object_index, finished]
for payload in events_by_block.get(total_blocks, ()): for event_sample, payload in events_by_block.get(total_blocks, ()):
self._apply_payload(payload) update = frame_update(payload)
if "object_blocks" in update and update["object_count"]:
self._schedule_payload(update, event_sample)
else:
self._current_payload = payload
self._apply_legacy(update)
for targets, ramp_samples in self._pending.pop(total_blocks, ()):
self._apply_meta_targets(targets, ramp_samples)
for block_index in sorted(self._pending):
for targets, ramp_samples in self._pending[block_index]:
self._apply_meta_targets(targets, ramp_samples)
self._pending.clear()
if standard_order: if standard_order:
return output[:, list(self.layout.internal_to_standard)] return output[:, list(self.layout.internal_to_standard)]
return output return output
@@ -313,10 +869,15 @@ class PythonSpeakerRenderer:
def render_objects16(objects16: np.ndarray, payload_events: Iterable[tuple[int, bytes]], def render_objects16(objects16: np.ndarray, payload_events: Iterable[tuple[int, bytes]],
layout: str | SpeakerLayout, *, block_size: int = 32, layout: str | SpeakerLayout, *, block_size: int = 32,
standard_order: bool = True, output: np.ndarray | None = None) -> np.ndarray: standard_order: bool = True, output: np.ndarray | None = None,
trim_meta=None, parameters=None, size_budget: int = 0) -> np.ndarray:
"""Render a complete interleaved LFE + 15 object stream.""" """Render a complete interleaved LFE + 15 object stream."""
renderer = PythonSpeakerRenderer(layout, block_size=block_size) renderer = PythonSpeakerRenderer(layout, block_size=block_size,
return renderer.process(objects16, payload_events, standard_order=standard_order, output=output) trim_meta=trim_meta, parameters=parameters,
size_budget=size_budget)
return renderer.process(objects16, payload_events, standard_order=standard_order,
output=output)
def payload_events_from_index(index, frame_count: int, *, frame_samples: int = 1536, def payload_events_from_index(index, frame_count: int, *, frame_samples: int = 1536,
payload_id: int = 11, outer_offset_samples: int = 1473): payload_id: int = 11, outer_offset_samples: int = 1473):
@@ -325,6 +886,7 @@ def payload_events_from_index(index, frame_count: int, *, frame_samples: int = 1
if payload_id in subpayloads: if payload_id in subpayloads:
timing = frame_update(subpayloads[payload_id]) timing = frame_update(subpayloads[payload_id])
yield ( yield (
frame * frame_samples + outer_offset_samples + timing["block_offset_samples"], frame * frame_samples + outer_offset_samples
+ int(timing["sample_offset"]),
subpayloads[payload_id], subpayloads[payload_id],
) )
+272
View File
@@ -0,0 +1,272 @@
"""Programme trim and front-back balance taken from the OAMD trim element.
TS 103 420 splits a programme's trim data into nine configurations, selected
by a layout's surround and height speaker counts:
cfg = kappa_f(surrounds) + 3 * kappa_h(heights)
kappa_f(n) = 0 for n == 0, 1 for n in (2, 3), 2 otherwise
kappa_h(n) = 0 for n == 0, 1 for n == 2, 2 otherwise
Each configuration holds one unit of calibration:
* a level in dB for the centre, surround and height speaker groups, and
* a front-back balance for the listener plane and for the overhead layer.
Only the layouts listed in ``SUPPORTED_LAYOUTS`` carry these semantics.
Configurations left at default contribute neither a static level nor a balance:
the automatic attenuation is already applied per object by the layout position
compensation. Configurations declared as disabled contribute nothing either.
The balance reparameterises the front-back axis; it does not move objects off
the axis and it leaves the two walls untouched. See ``front_back``.
"""
from __future__ import annotations
from dataclasses import dataclass
import math
from variant_error import UnsupportedVariantError
# Layouts that carry trim and front-back balance semantics, by their compact
# names: 2.0, 3.1, 4.0, 5.1, 5.1.2, 5.1.4 and 7.1. Wider layouts leave the
# programme untrimmed.
SUPPORTED_LAYOUTS = ("20", "31", "40", "51", "512", "514", "71")
# Channel label -> trim group. Left, right and LFE carry no trim in any
# configuration, so they are absent from the table.
GROUPS = {
"C": "centre",
"Ls": "surround", "Rs": "surround",
"Lrs": "surround", "Rrs": "surround",
"Ltf": "height", "Rtf": "height",
"Ltr": "height", "Rtr": "height",
"Ltm": "height", "Rtm": "height",
}
FLOOR_LABELS = ("Ls", "Rs", "Lrs", "Rrs")
HEIGHT_LABELS = ("Ltf", "Rtf", "Ltr", "Rtr", "Ltm", "Rtm")
# bal3d_y is coded in steps of 1/16, so a balance is quantised to that grid.
BALANCE_STEPS = 16
# Front-back balance keeps a 5 percent wall band at each end and leaves it unmapped.
WALL = 0.05
@dataclass(frozen=True)
class SpeakerRenderParameters:
"""Codec-independent, per-layout speaker controls; balances are in [-1,1].
Warp 0 renders directly, 1 scales floor Y, and 2/3 render through an
expanded surround layout before PLIIx/Lo/Ro folding. An absent warp
indication uses normal (0); no CLI override is provided.
"""
warp_mode: int = 0
trim_mode: str = "automatic"
surround_db: float = 0.0
height_db: float = 0.0
centre_db: float = 0.0
listener_balance: float = 0.0
overhead_balance: float = 0.0
disable_per_object: tuple[bool, ...] = ()
def __post_init__(self):
if self.warp_mode not in (0, 1, 2, 3):
raise ValueError(f"invalid warp mode: {self.warp_mode!r}")
if self.trim_mode not in ("automatic", "disabled", "explicit"):
raise ValueError(f"invalid trim mode: {self.trim_mode!r}")
for value in (self.surround_db, self.height_db, self.centre_db):
if not math.isfinite(value):
raise ValueError("trim levels must be finite")
for value in (self.listener_balance, self.overhead_balance):
if not math.isfinite(value) or not -1.0 <= value <= 1.0:
raise ValueError("balance must be finite and in [-1,1]")
def bypasses(self, object_index):
return (0 <= object_index < len(self.disable_per_object)
and self.disable_per_object[object_index])
def parameters_from_oamd(atmos_meta, layout):
"""Adapt OAMD calibration to one layout, without changing source positions.
A config's default/disabled flag overrides global custom mode. Missing
custom configuration falls back to automatic.
"""
meta = atmos_meta or {}
common = dict(warp_mode=meta.get("warp_mode", 0),
disable_per_object=tuple(meta.get("disable_per_obj") or ()))
if not _supported(layout):
return SpeakerRenderParameters(**common, trim_mode="disabled")
mode = meta.get("global_trim_mode", 0)
if mode == 1:
return SpeakerRenderParameters(**common, trim_mode="disabled")
if mode not in (0, 1, 2):
raise UnsupportedVariantError(
"speaker", "trim_mode_not_implemented", "Reserved OAMD global trim mode",
details={"global_trim_mode": mode})
record = _record(meta, layout) if mode == 2 else None
if not isinstance(record, dict) or record.get("default"):
return SpeakerRenderParameters(**common)
if record.get("disabled"):
return SpeakerRenderParameters(**common, trim_mode="disabled")
# Surround/height are attenuation-only. Centre retains its signed level.
return SpeakerRenderParameters(
**common, trim_mode="explicit",
surround_db=min(float(record.get("surround", 0.0)), 0.0),
height_db=min(float(record.get("height", 0.0)), 0.0),
centre_db=float(record.get("centre", 0.0)),
listener_balance=float(record.get("bal3d_y_lis", 0.0)),
overhead_balance=float(record.get("bal3d_y_tb", 0.0)))
def explicit_position_gain(parameters, x, y, z):
"""Position-dependent *object* trim, evaluated at balance(original Y).
Surround fades in over Y=0..0.5 and out towards Z=1; height fades in
with Z. Centre trim is local to the front-centre region. The resulting
scalar multiplies the object's entire speaker gain vector, before mixing.
"""
surround = min(max(2.0 * y, 0.0), 1.0) * (1.0 - z)
centre = 1.0 - min(max(10.0 * max(
abs(z) - WALL, y - WALL, abs(x - 0.5) - WALL), 0.0), 1.0)
db = (surround * min(parameters.surround_db, 0.0)
+ z * min(parameters.height_db, 0.0)
+ centre * parameters.centre_db)
return 10.0 ** (db / 20.0)
def _height_class(count):
if count == 0:
return 0
if count == 2:
return 1
return 2
def _floor_class(count):
if count == 0:
return 0
if count in (2, 3):
return 1
return 2
def layout_classes(layout):
"""Return ``(config_index, height_count, floor_count)`` for a layout."""
channels = tuple(getattr(layout, "channels", ()))
heights = sum(1 for c in channels if c in HEIGHT_LABELS)
floor = sum(1 for c in channels if c in FLOOR_LABELS)
return _floor_class(floor) + 3 * _height_class(heights), heights, floor
def _supported(layout):
return getattr(layout, "name", None) in SUPPORTED_LAYOUTS
def _record(atmos_meta, layout):
if not atmos_meta or not _supported(layout):
return None
index, _, _ = layout_classes(layout)
configs = atmos_meta.get("configs") or ()
if index >= len(configs):
return None
return configs[index]
def group_levels_db(atmos_meta, layout):
"""Trim level in dB for the centre, surround and height speaker groups.
Only an explicit configuration contributes a static level. An automatic
configuration carries no static gain: the attenuation it describes is
already applied per object by the layout position compensation, and
multiplying it in a second time would count it twice.
"""
record = _record(atmos_meta, layout)
if not isinstance(record, dict) or record.get("default") or record.get("disabled"):
return {"centre": 0.0, "surround": 0.0, "height": 0.0}
return {
"centre": float(record.get("centre", 0.0)),
"surround": float(record.get("surround", 0.0)),
"height": float(record.get("height", 0.0)),
}
def channel_gains(atmos_meta, layout):
"""Legacy diagnostic group levels, not the speaker rendering operation.
Rendering uses explicit_position_gain per object before accumulation.
"""
levels = group_levels_db(atmos_meta, layout)
gains = []
for label in getattr(layout, "channels", ()):
group = GROUPS.get(label)
db = levels[group] if group else 0.0
gains.append(10.0 ** (db / 20.0))
return gains
def balance_layers(atmos_meta, layout):
"""Front-back balance for the listener plane and the overhead layer.
Both are returned as signed magnitudes in [-1, 1]; zero means no balance.
Only an explicit configuration carries a balance.
"""
record = _record(atmos_meta, layout)
if not isinstance(record, dict) or record.get("default") or record.get("disabled"):
return 0.0, 0.0
listener = float(record.get("bal3d_y_lis", 0.0))
overhead = float(record.get("bal3d_y_tb", 0.0))
return listener, overhead
def _curve(balance):
"""Segment coefficients for one balance value.
Returns ``(a0, a1, a2, a3)`` for the two halves of the axis, or ``None``
when the balance is zero.
Writing ``s = |n| / 18`` with ``n`` the balance quantised to the coding
grid, a positive balance stretches the front half by ``1 + s`` and
compresses the back half by ``1 - s``; a negative balance does the
reverse. Both halves anchor at the axis midpoint and at the two walls,
so the mapped value always stays inside the wall range.
"""
n = int(round(abs(balance) * BALANCE_STEPS))
if n == 0:
return None
s = n / 18.0
if balance > 0:
return 1.0 + s, -WALL * s, 1.0 - s, 0.5 + 0.45 * s
return 1.0 - s, WALL * s, 1.0 + s, 0.5 - 0.45 * s
def front_back(position, balance):
"""Reparameterise a front-back coordinate in [0, 1] by ``balance``.
``0`` is the front wall and ``1`` the back wall. Positions inside the
wall bands are returned unchanged, and so is every position when the
balance is zero.
"""
curve = _curve(balance)
if curve is None:
return position
if position <= WALL or position >= 1.0 - WALL:
return position
a0, a1, a2, a3 = curve
if position < 0.5:
return a0 * position + a1
return a2 * (position - 0.5) + a3
def apply_balance(position, height, listener, overhead):
"""Blend the two balance layers by height and map the coordinate.
``height`` runs from 0 at the listener plane to 1 at the top layer.
"""
if not listener and not overhead:
return position
low = front_back(position, listener)
high = front_back(position, overhead)
return (1.0 - height) * low + height * high