Multi Language Audio with Live Streams

Introduction

This feature allows ingesting multiple audio languages, descriptive or ambient audio experience tracks to be available for live playback.

Prerequisites

For details about the following values, refer to your encoder documentation:

• Define the PID (Packet Identifier) for each audio track in your encoder
• Define the Video PID asscociated with the video in your encoder

If you don't know the PID values from your encoder, see the Getting encoder values section.

Adding Audio Tracks

When creating a new Live event or channel:

1. In Video Cloud Studio, navigate to the Live module.

2. Click either Create Channel or Create Event.

   

3. Expand the Audio Tracks section. You may see that the Add Audio Tracks button is grayed out. This is because multiple audio tracks are only supported for RTP and SRT input formats.

   

4. Expand the Advanced Options section and change the Input Format to either RTP or SRT. Now, the Add Audio Tracks button will be enabled.

   

5. For CIDR Whitelist, enter either 0.0.0.0/0 to connect to the RTP endpoint to allow all networks, or a specific value. This controls who can connect to the RTP or SRT endpoints.

   

6. For more information about advanced input formats, see the Configuring advanced options document.

7. Click Add Audio Tracks.

   

8. Enter field values for each audio track. Fields with an asterisk (*) are required.

   

   • Default - Select the default audio track
   • Language* - The language code for the audio track; Example: en, es, cy-GB, etc.
   • PID* - The Packet Identifier (PID) code defined in your encoder to sync audio tracks in your incoming stream to the correct language and variant
   • Video PID* - The Packet Identifier (PID) code associated with the video; this value is the same for all audio tracks
   • Variant - The variant of the audio track; values are:
     • main
     • alternate
     • caption
     • subtitle
     • supplementary
     • commentary
     • dub
   • Label - It is highly recommended to enter a label value (example: English, Spanish, etc.); if the label field is blank, the language code will be used as the label
9. Click the + and - buttons, to add or remove audio tracks.
10. Fill in the rest of the fields for a new channel or event. For details, see the Creating and Managing Live Events document.
11. Click either Create Event or Create Channel.

Switching Audio Track

1. If the multi-audio tracks were set up correctly, the player will have a headphone icon that can be used to switch between tracks.

   

Getting encoder values

For the following fields, you will provide values from your encoder:

• PID - The packet idenifier from the MPEG-TS that refers to a specific audio track
• Video PID - The packet idenifier from the MPEG-TS input stream to select the video track

Example with ffprobe

If you don't know the PIDs from your encoder, you can use a tool such as ffprobe (included with ffmpeg) to inspect the PIDs in the stream.

ffprobe /tmp/ma.ts
ffprobe version 5.0.1 Copyright (c) 2007-2022 the FFmpeg developers
  built with Apple clang version 13.0.0 (clang-1300.0.29.3)
  configuration: --prefix=/usr/local/Cellar/ffmpeg/5.0.1-with-options_1 --enable-shared --cc=clang --host-cflags= --host-ldflags= --enable-gpl --enable-libaom --enable-libdav1d --enable-libmp3lame --enable-libopus --enable-libsnappy --enable-libtheora --enable-libvorbis --enable-libvpx --enable-libx264 --enable-libx265 --enable-libfontconfig --enable-libfreetype --enable-frei0r --enable-libass --enable-demuxer=dash --enable-opencl --enable-audiotoolbox --enable-videotoolbox --disable-htmlpages --enable-libvmaf --enable-version3
  libavutil      57. 17.100 / 57. 17.100
  libavcodec     59. 18.100 / 59. 18.100
  libavformat    59. 16.100 / 59. 16.100
  libavdevice    59.  4.100 / 59.  4.100
  libavfilter     8. 24.100 /  8. 24.100
  libswscale      6.  4.100 /  6.  4.100
  libswresample   4.  3.100 /  4.  3.100
  libpostproc    56.  3.100 / 56.  3.100
Input #0, mpegts, from '/tmp/ma.ts':
  Duration: 00:00:04.86, start: 1.400000, bitrate: 502 kb/s
  Program 1
    Metadata:
      service_name    : Service01
      service_provider: FFmpeg
  Stream #0:0[0xff]: Video: h264 (Constrained Baseline) ([27][0][0][0] / 0x001B), yuv420p(progressive), 360x240 [SAR 1:1 DAR 3:2], 30 fps, 30 tbr, 90k tbn
  Stream #0:1[0x100]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 98 kb/s
  Stream #0:2[0x102]: Audio: aac (LC) ([15][0][0][0] / 0x000F), 44100 Hz, mono, fltp, 98 kb/s

The hex number after the stream will indicate the PID. For example:

• 0xff == 255
• 0x100 == 256
• 0x101 == 257

Using the Live API

You can also use the Live API to manage multiple language audio tracks with your live streams.

Example audio track

Here's an example of an alternate audio track:

{
  "alternate_audio": {
    "tracks": [
      {
        "label": "English",
        "language": "en",
        "variant": "main",
        "video_pid": 256,
        "pid": 257,
        "default": true
      },
      {
        "label": "Spanish",
        "language": "es",
        "variant": "main",
        "video_pid": 256,
        "pid": 258
      }
    ]
  }
}