u/Whole_Willow_5027

Hi everyone,

I’m a software engineer who recently transitioned into a role working on media playback for embedded systems and smart TVs.

To clarify my scope, I am strictly working at the software/application layer. I am not designing the firmware or working on the hardware, but I’d like to not treat it as a “black box.”

I’m trying to understand the fundamental architecture of the playback pipeline: how a manifest is read, buffering, demuxing, passing bits to hardware decoders, and rendering to the screen (and anything else I may have missed)

The problem is, I feel like I’ve read a ton of documentation and resources, but the concepts are still incredibly foreign to me. I’m having a really hard time building a solid mental model of how all these moving parts connect.

Any advice on the best pedagogical approach to learning about media processing pipelines? Is there a standard university textbook, a specific project based course, or a teaching method that makes these concepts click? How is this typically taught at the graduate level to engineers? I only did my undergrad and this unfortunately was something we never touched on. Thank you in advance!

Textbooks I’ve come across so far, but have struggled to get past the first few chapters:

Video Demystified by Keith Jack

How Video Works by Marcus Weise and Diana Weynand

The Art of Digital Video by John Watkinson

Embedded media processing by David Katz

Hi folks,

I’m a software engineer who recently transitioned into a role working on media playback for embedded systems and smart TVs.

To clarify my scope, I am strictly working at the software/application layer. I am not designing the firmware or working on the hardware, but I’d like to not treat it as a “black box.” I need a solid mental model on how the underlying system actually processes what my software sends it.

I’d like to get up to speed on the pipeline: fetching chunks, demuxing, feeding the decoders, managing buffers, and rendering frames (and anything else I may have missed).

I’ve been reading a lot of resources, but to be honest, the concepts are so foreign that I’m struggling to conceptualize how the system actually flows.

Any advice on the best approach to learning this domain? Should I be playing around with Gstreamer pipelines? Building a basic player from scratch? If anyone knows of a structured course, a great textbook, or just hands-on learning strategies that helped you build a mental model of embedded media playback, I would really appreciate it. Thanks in advance!

Textbooks I’ve come across so far, but have struggled to get past the first few chapters:

Video Demystified by Keith Jack

How Video Works by Marcus Weise and Diana Weynand

The Art of Digital Video by John Watkinson

Embedded media processing by David Katz

Hi everyone,

I recently started a software engineering role that involves porting a video application to embedded devices and smart TVs. I come from a standard SWE background so I’m completely new to the video engineering side.

I’m trying to understand the fundamental architecture of the playback pipeline: how a manifest is read, buffering, demuxing, passing bits to hardware decoders, and rendering to the screen (and anything else I may have missed)

The problem is, I feel like I’ve read a ton of documentation and resources, but the concepts are still incredibly foreign to me. I’m having a really hard time building a solid mental model of how all these moving parts connect.

What was your learning approach (especially for those of you who have made the jump to video engineering)? Did you build a simple project? Are there specific visual resources, courses, or textbooks that finally made the concepts “click” for you? I’m looking for advice on how to study this effectively along with resource recommendations. Thank you in advance!

Textbooks I’ve come across so far, but have struggled to get past the first few chapters:

- Video Demystified by Keith Jack

- How Video Works by Marcus Weise and Diana Weynand

- The Art of Digital Video by John Watkinson

- Embedded Media Processing by David Katz

Edit: to clarify, I am strictly working on the software side. I don’t touch the hardware/firmware directly. I just use the APIs that do, but it would be helpful to understand how it all works.