Practical considerations for real-time super-resolution implementation techniques over video coding platforms

Abstract

This paper addresses practical considerations for the implementation of algorithms developed to increase the image resolution from a video sequence by using techniques known in the specialized literature as super-resolution (SR). In order to achieve a low-cost implementation, the algorithms have been mapped onto a previously developed video encoder architecture. By re-using such architecture and performing only slight modifications on it, the need for specific, and usually high-cost, SR hardware is avoided. This modified encoder can be used either in native compression mode or in SR mode, where SR can be used to increase the image resolution over the sensor limits or as a smart way to perform electronic zoom, avoiding the use of high-power demanding mechanical parts. Two SR algorithms are presented and compared in terms of execution time, memory usage and quality. These algorithms features are analyzed from a real-time implementation perspective. The first algorithm follows an iterative scheme while the second one is a modified version where the iterative behavioural has been broken. The video encoder together with the new SR features constitutes an IP block inside Philips Research, upon which several System on Chip (SoC) platforms are being developed.