Streams and collections: we're not done yet!

Decodebin3 and playbin3 brought a more efficient handling of playback use-cases by explicitly listing available streams, allowing fast stream-switching (by not decoding all streams), and a leaner codebase. The core feature for allowing this was the addition to GStreamer of collections of GstStream (i.e. explicit listing of streams).

This talk will go over proposed additions to the streams API to go the extra mile and allow use-cases that weren't possible before or weren't efficient:

• stream-selection by any element (as opposed to just decodebin3). This will allow elements such as adaptive demuxers to only download the streams really required (as opposed to all streams).
• reliably notify elements that a given stream won't be used at all downstream (to reduce resource usage even more)
• know as early as possible when elements are ready to receive processing instructions, such as seek events or stream-selection, instead of waiting for pre-rolling.
• handle scalable streams (where the base and enhancement layers are separate) such as SHVC, Dolby Vision, wavpack, and more.

Edward Hervey has been contributing to the GStreamer project for the past 14 years, from core components to high-level projects such as the pitivi video editor. Currently a Senior Engineer at Centricular, he has helped numerous clients in current and past companies to make the most out of GStreamer in various areas. He is currently in charge of Continuous Integration and overseeing QA infrastructure for the GStreamer project.