Large scale distributed video processing with OTP

View Slides

Video processing is both CPU and IO heavy. When it comes to Tubi, where the system need to process thousands of hours each week, it became a quite big challenge.

The Tubi team has created a way of breaking a single video process job to hundreds of tasks and distributed different tasks to different nodes that live in the same cluster. The cluster is built on OTP, the resources (CPU and IO) used by each task on each node are properly managed. Each task is doing a tiny bit of the whole thing, the input of each task may come from the output the task, both tasks may be running in different nodes.

THIS TALK IN THREE WORDS

Video processing

OTP

OBJECTIVES

Share the system we built which connects video processing with OTP.

TARGET AUDIENCE

All Elixir/Erlang engineers and people who are into video related things.

Haofei Wang

Back to all media