When watching a livestream of a conversation between multiple participants in distant locations, a viewer might observe a delay when the speaker changes. This delay is caused by network latency between the speakers and results in “dead air” in the broadcast - when no one is speaking, therefor hurting the flow of the conversation. This project’s goal is to implement a video editing system that would eliminate this phenomenon by stretching the stream, accumulating video packets, and cutting an unwanted “gap”
The system is able to stretch the stream according to a given stretch function, which fills up a buffer of video frames and audio packets that the system uses to cut dead-air parts of the dialog when switching from one participant to another. Stretching the stream introduces a new problem- a deformed pitch, which needs to be corrected to make the stretching as seamless as possible and preserve the stream quality.
This system can be used in any live stream program where the described problem is noticed, such as news reports, online streaming, video chats, and so on.
The system is implemented using Blackmagicdesign decklink devices, and Blackmagicdesign’s SDK. These video acceleration devices that connect to the host machine via PCIe and are accessible via SDI cables, and mimic a live stream, like a news report for example. The devices allow us to perform software manipulations before outputting the stream to the spectators at home, represented by a monitor on which we display the result video.
The algorithm takes a party of N speakers and is meant to let one participant speak at a time, while not letting unwanted dead-air frames appear on the end-point external monitor.
