Immersive Audio (IMAU) Premium Immersion is an up-mixing algorithm that converts a 2-channel input signal to an 12-channel output signal to improve spaciousness and immersiveness experience. This is a 7.1.4 output configuration, where 4 height speakers are supported to produce a 3D effect. 

Objectives:

1. Producing a natural, artifact-free sound.
2. Low MIPS and memory consumption
3. Flexible design

Output Layout

Channel No	Output
1	Left Front Speaker
2	Right Front Speaker
3	Center Speaker
4	Sub-Woofer Speaker
5	Left Side Speaker
6	Right Side Speaker
7	Left Rear Speaker
8	Right Rear Speaker
9	Left Height Front Speaker
10	Right Height Front Speaker
11	Left Height Rear Speaker
12	Right Height Rear Speaker

 

Main Processing Blocks

IMAU Premium Immersion is built with several processing blocks:

• Spectral Manager: splits the source signal into 2 bands (Low Band and High Band)
• Center Extraction: extracts two sound components from the native stereo signal (Extracted Center and Residuals Left and Right)
• Ambience: generates Early Reflection and Reverb to simulate room acoustics effects
• Mixing and Routing: mixes all the generated signals to create an immersive sound field.

The Custom Panel is arranged in two sections 

1. The area on the left-hand side hosts 5 tabs. Each tab is dedicated to control and tune a specific feature or group of features, as per the following arrangement:
   • SPECTRAL MANAGER
   • SPREAD
   • EARLY REFLECTIONS
   • REVERB
   • SUPERUSER
2. The section on the right-hand side is called Master Control and is reserved to adjust several master functions.

The Master Control Section remains always visible and accessible, while the other features can be accessed individually via the corresponding tab.

Spectral Manager

IMAU Premium Immersion integrates a 2-bands Spectral Manager designed to ensure that the whole audio spectrum is optimally reproduced in the car.

Spectral Manager takes the full-band stereo input signal and splits it into the following 2 bands:

• Low Band (in the range 40-200 Hz)
• High Band (from minimum 40-200 Hz to half of the sampling rate)

The Spectral Manager is also complemented by the Extracted Center Band Split functionality where the High Band of the Center channel is further split into two bands by a crossover filter. The lower frequency component is routed to the Front channels and the higher frequency component is routed to the Center channel.

In the SPECTRAL MANAGER tab, users can control the Equalizers and the Gains AOs integrated in IMAU Premium Immersion. Specifically, they can:

• Enable/disable 2-Band Separation (A)
• Set the crossover frequency of the high-pass and low-pass filters used to generate the band-limited signals (B),
• Set the gain for each Low Band signal (C),
• Independently mute/unmute each Low Band signal (D),
• Enable/disable the gain applied to the lower frequency Center component (E),
• Set the gain for the lower frequency center component (F),
• Set the crossover frequency for the high-pass and low-pass filters used to generate the higher and lower frequency Center components (G).

Spread

One of the main features implemented in IMAU Premium Immersion consists in separating the stereo source signal into two components, namely Extracted Center (A) and Residuals (B).

Extracted Center is a 1-ch signal generated from the Left and Right stereo signals having same frequency, energy, and phase.

Residuals is the 2-ch signal remaining after removing the Extracted Center from the stereo source.

The SPREAD tab exposes the parameters to control how these components are processed and routed to the 7.1.4 output.

Once Extracted Center and Residuals signals are generated, they are split into multiple copies. Each copy is independently routed to a specific loudspeaker (or loudspeakers pair) to create the immersive sound field, as per the following grouping (C):

TO MAIN CH

• Center (Center channel)
• Front (Left and Right Front channel)
• Side (Left and Right Side channel)
• Rear (Left and Right Rear channel)

TO HEIGHT CH

• Front (Left and Right Height Front channel)
• Rear (Left and Right Height Rear channel)

The lower area of the SPREAD tab hosts several fields for configuring the Gain ranges of the Smart Tuning Parameters logic

Ambience

In IMAU Premium Immersion, the immersive sound field creation is enhanced by additional signals generated to increase the spaciousness perception. This effect is produced by the signal flow belonging to the AMBIENCE section in the block diagram. It includes two processing blocks, namely, Early Reflections and Reverb. The input to the AMBIENCE processing section is the High Band signal of the native stereo input, as generated by Spectral Manager.

Early Reflections

The EARLY REFLECTIONS tab is divided into two sections, SURROUND CHANNELS (A) and HEIGHT CHANNELS (B), and exposes some parameters to control the Early Reflections settings, including:

• ON/OFF switches, to enable/disable Early Reflection signals generation (G) or to mute/unmute individual channel group (I),
• Vertical faders, to set the global (F) and the separate (E) gains of the signals routed to the Surround channels (ch.1&2 are routed to the Side channels and ch.3&4 to the Rear channels) and the Height channels (ch.1&2 are routed to the Height Front channels and ch.3&4 to the Height Rear channels),
• Vertical faders, to set the global (C) and the separate (D) delays of the signals routed to the Surround channels (ch.1 is routed to the Left Side channel, ch.2 is routed to the Right Side channel, ch.3 is routed to the Left Rear channel and ch.4 is routed to the Right Rear channel) and the Height channels (ch.1 is routed to the Left Height Front channel, ch.2 is routed to the Right Height Front channel, ch.3 is routed to the Left Height Rear channel and ch.4 is routed to the Height Rear channel),
• An ON/OFF switch and a knob (H) to enable/disable and control the cut-off frequency of the Low-Pass filter.

Static gain can be applied to each signal’s copy via vertical faders (D). These signals can also be independently muted and unmuted (E).

Each signal is further processed by a 2-Biquad filter. Via the EQ buttons (F), users can launch the Native Panels of those filters and configure their tuning (F).

The lower area of the EARLY REFLECTIONS tab hosts the fields for configuring the gain ranges of the Smart Tuning Parameters logic.

Reverb

In IMAU Premium Immersion, the immersive sound field creation is enhanced by a reverb processing to increase the spaciousness perception. Routing is divided into SURROUND CHANNELS (A) and HEIGHTS CHANNELS (B). The amount of reverb is given by the GAIN faders (C), also individual routing to side and to rear can be adjusted in the connected parameters sub menu (E) that can be opened in the left side of the fader. Reverb can be turned ON and OFF completely in the main ON/OFF button (F) or be partially turned ON and OFF in different channels (G). There is a pre delay control (H) in the HEIGHT CHANNELS (B) to separate the processed signal from the surround if it is needed.

IMPORTANT NOTE The REVERB CONTROLS parameters should be tuned carefully. In particular, the ambience effect (reverb and early reflections) should be used very subtly. The goal is that the listener does not perceive an additional room (i.e., long reverberation, resonances, changed timbre) but rather that additional reflections expand the perceived spatiality in a neutral way. It is therefore recommended to make the reverb short (Time) and narrow band (Low-Pass and High-Pass)

REVERB CONTROLS (D):

• Time: affects feedback damping and output mixing so that a small RT factor resembles a small room, and a large factor approaches infinite reverb time.
• Low-Pass: defines the amount of high frequencies of the reverb by means of first-order low-pass filter.
• High-Pass: defines the amount of low frequencies of the reverb by means of first-order high-pass filter.
• WET ONLY: When switched on, only the generated reverb is present at the output. When switched off, the input signal is mixed into the output.
• EQ: button to launch the Native Panel of the equalizer applied to the Reverb output signals.

Superuser

The fifth tab is called SUPERUSER, showed in Figure 7, and exposes controls of the following processing functions (detailed parameter description follows below). Superuser is divided into 3 sections:

1. DECORRELATION (A)

               Decorrelation can be applied to Surround channels (D) and to Height channels (E) independently, with different IR filters. Also, independent ON/OFF buttons control each group of channels (F).

2. HEIGHT CHANNELS (B)

               In this section, there are 2 faders to apply delay to the Height channels with a maximum of 5 ms, these delays are applied only to the Extracted Center and the Residual signals routed to the Height channels. One fader applies delay to the front (G), and one to the rear (H), independently. The Rear channels mirroring (I) invert left to right and right to left in the Height Rear channels.

3. MONO DETECTOR (C)

        Mono detector can be enabled or disabled with the ON/OFF button (J), and it has 2 controls, Time to detect Mono (K), this is the amount of time that will take to turn off the Early Reflections and Reverb when a mono signal is detected, and the Time to detect Stereo (L), which is the amount of time that it will take to return to immersive mode.

Master Control Section

The Master Control Section exposes controls for:

• Defining offset gains for HEIGHT CHANNELS (Front, Rear) and LOW CHANNELS (Front, Side, Rear) (A),
• Setting global output Master Level (B),
• Shaping the output 7.1.4 signal based on the Smart Tuning Parameters ranges configured in Spread, Early Reflections and Reverb (C),
• Muting output channels or channels groups (D),
• Launching the Native Panel of the Matrix Mixer AO integrated in IMAU for routing to the 7.1.4 output channels all signals generated during the upmixing process (E),
• Recalling, storing, and deleting presets (F).

Smart Tuning Parameters (STP)

IMAU Premium Immersion integrates a method for allowing seamless transitioning between two presets. This approach is called Smart Tuning Parameters (STP) and is realized via a signal flow based on LUT controlled via custom Control ID called Immersiveness. The Immersiveness Control ID is exposed to the Custom Panel in the Master Control Section.

It ranges from MIN and MAX values.

The actual MIN and MAX values can be set in the SPREAD, EARLY REFLECTIONS and REVERB tabs (see Spread, Early Reflections and Reverb). They appear in TextBox cells with black background displayed in the lower parts of each tab.

 

IMPORTANT NOTE The Smart Tuning Parameters logic applies only to the STP parameters: –         Gains of Extracted Center and Residuals signals (SPREAD tab) –         Gains of Early Reflections (EARLY REFLECTIONS tab) –         Gains of Reverb (REVERB tab) No other parameters are affected by the Immersiveness knob setting (non-STP parameters).

 

Configuring STP

The following procedure shall be executed to properly configure the STP tuning.

1. Tune the non-STP to obtain an average immersive sound field,
2. Set Immersiveness to MIN,
3. Tune the STP to obtain a minimum immersive audio output,
4. Manually type in the values of the STP into the corresponding MIN fields in SPREAD, EARLY REFLECTIONS and REVERB tabs,
5. Set Immersiveness to MAX,
6. Tune the STP to obtain a maximum immersive audio output,
7. Manually type in the values of the STP into the corresponding MAX fields in the SPREAD, EARLY REFLECTIONS and REVERB tabs,
8. Use the Immersiveness knob to seamlessly move between the minimum and the maximum immersive output,
9. Amend the non-STP to optimize the result for any Immersiveness

IMPORTANT NOTE It should be noted that the values in STP sliders need not always correspond to the actual values being applied. This is because the actual values (which can be observed in the state variable of the corresponding AO) depend on the position of the Immersiveness knob and the MIN and the MAX values of STP. The sliders are also not updated with the actual values being set by the Immersiveness knob by design. If needed, STP can be disabled by not sending the control ID information when sending the signal flow. That way the values of the sliders are directly applied to the Volume and Mute AOs.

 

This section describes some guidelines to help the user operate in the most efficient way for the most effective experience using IMAU Premium Immersion.

Instantiating IMAU Premium Immersion CAO

To instantiate IMAU Premium Immersion CAO in GTT follow these operations:

• in Device Designer, Toolbox, import the .CAO file (e.g.: IMAU Premium 1.0.CAO) (A),
• drag & drop IMAUPremiumImmersion from the Compound Audio Objects dropdown menu into the signal flow area (B),

• instantiate a ControlIn AO (C) and configure it to 1 channel outputs (D),
• connect the ControlIn output pin to IMAUPremiumImmersion Control input pin.

• in Device View, press Control IDs (E) and then Import (F),

• select the IMAU Premium Control IDs.csv file and import it. The Custom Control IDs popup window will show the following imported Control ID:

In Device Designer:

• select the ControlID AO and add 1 Control In (G),
• assign Immersiveness to Pin 1 (H),
• Save,
• Edit Device,
• Update Device

In Device Designer:

• Select IMAUPremiumImmersion CAO and add CAO Custom Panel to the list (I)

In Custom Panel:

• Drag&drop the Immersiveness Control ID (J) to link it to the IMMERSIVENESS knob in the Master Control Section of the IMAU Premium Immersion panel. Replace the parameter when requested.

In Device Designer:

• Select IMAUPremiumImmersion CAO and tick the Is Custom Panel checkbox (L)

Creating Immersive Sound Fields

The perception of a realistic immersive sound field is enhanced when the localization of sound sources is easier for the listener to identify.

On the other hand, room acoustics cues generated as soundwaves reflected by the environment surrounding the listener help significantly to determine the perception of being immersed in a real ambience.

To improve the effectiveness of the upmixing process it is very important to define a realistic balance between discrete signal distribution and homogeneous ambience acoustic feedback.

Therefore, it is good practice to aim at differentiating the signals reproduced by each loudspeaker (e.g., avoiding routing the same signal and with the same energy to more than one loudspeaker or loudspeakers group) whilst gluing them via an appropriate dose of reflexed sound field as provided by artificial ambience simulations tools like delays and reverberators.

“Front Stage” and “On Stage” Tuning Strategies

With the IMAU Premium Immersion approach, different tuning philosophies can be realized. Two well-known ones are “Front Stage” (also known as “Audience” in QLS) and “On Stage”, which are described below.

“Front Stage”: The goal of this tuning is that the stage (i.e. the musicians, instruments or more generally: the sound events) is in front of the listener and from behind only a natural spatiality is perceived that acoustically matches the sound event on the stage.

Tuning strategy: Clean, stable stage that comes from the front. Side channels are used to carefully insert the residuals to widen the stage slightly. Little residuals from the rears. Overall little energy from side/rear channels.

“On Stage”: With this tuning, the goal is to distribute the laterally placed sound events in the original mix to the sides or rear channels. It creates the impression for the listener that they are on stage.

Tuning strategy: Significantly more residuals from the sides and rear channels. This means that the partially uncorrelated signal components are automatically placed on the side channels (which also contain components of the extracted center signal) and the completely uncorrelated (hard panned) components are placed on the rear channels (which normally should not contain any components of the extracted center signal).