Semantic Communication Based Video Coding Using Temporal Prediction of Deep Neural Network Parameters

Samarathunga Liyanage Don, Buddhi Prabhath Samarathunga and Ganearachchi, Yasith Lakmal and Fernando, Thanuj and Alahapperuma, Indika and Fernando, Anil; (2024) Semantic Communication Based Video Coding Using Temporal Prediction of Deep Neural Network Parameters. In: 2024 IEEE CTSoc Gaming, Entertainment and Media (GEM) Conference. 2024 IEEE Gaming, Entertainment, and Media Conference (GEM) . IEEE, ITA. ISBN 979-8-3503-7453-7 (https://doi.org/10.1109/GEM61861.2024.10585783)

[thumbnail of Semantic-Communication-Based-Video-Coding-Using-Temporal-Prediction-of-Deep-Neural-Network-Parameters]
Preview
 Text. Filename: Semantic-Communication-Based-Video-Coding-Using-Temporal-Prediction-of-Deep-Neural-Network-Parameters.pdf
Accepted Author Manuscript
License: Creative Commons Attribution 4.0 logo
Download (1MB)| Preview

Abstract

Video coding is a critical capability that underpins gaming, entertainment and media ecosystems, enabling effective use of video content in both conventional and non-conventional formats. Semantic communications, where semantics alone can be used to reconstruct media content provided that the context of semantic extraction is known, can effectively implement video coding, but techniques to exploit temporal correlations between video frames to achieve better rate distortion performance with them are just beginning to evolve. A novel approach for this problem of predicting the semantic decoder parameters using temporal correlation is proposed and tested using an autoencoder-based semantic communication system, and the performance is compared with the Neural Network Encoder- Decoder (NNCodec). Experimental results show that it achieves significantly better rate distortion performance compared to NNCodec alone, with PSNR gains between 3 and 25 dB depending on the complexity of the video and an average bitrate saving of 54%.

ORCID iDs

Samarathunga Liyanage Don, Buddhi Prabhath Samarathunga, Ganearachchi, Yasith Lakmal ORCID logoORCID: https://orcid.org/0000-0002-8337-3739, Fernando, Thanuj, Alahapperuma, Indika and Fernando, Anil;
CORE (COnnecting REpositories)