A Real Time FHD Learning Based Super Resolution System Without a Frame Buffer
Abstract : This brief presents a real-time learning-based superresolution (SR) system without a frame buffer. The system running on an Altera Stratix IV field programmable gate array can achieve output resolution of 1920 × 1080 (FHD) at 60 fps. The proposed architecture performs an anchored neighborhood regression algorithm that generates a high-resolution image from a low-resolution image input using only numbers of line buffers. This real-time system without a frame buffer makes it possible to integrate SR operation into image sensors or display drivers carrying out computational photography and display.
? In order to upgrade existing videos without extra storage costs, we propose an FPGA-based super-resolution system that enables real-time Ultra-HD upscaling in high quality.
? Our super-resolution system generates a higher resolution video than reported in existing literature, namely 3940×2160 UHD videos from 1920×1080 FHD sources at a frame rate of approximately 30fps on an embedded FPGA board.
? The existing super-resolution works and techniques, we proposed a real-time UHD super-resolution solution based on FPGA accelerator.
? Many accelerators focus on improving the computational efficiency.
? It is a well-known, ill-posed problem since a single HR image could generate more than one LR image, and it requires enough prior knowledge to reconstruct the highquality HR images.
? This problem, learning-based single-image SR methods have achieved outstanding performance and gained state-of-the-art results, by learning from millions of external image patches.
? This problem, the proposed SR architecture doubles the operation period in the second stage, and the system is then designed with multiple clock domains.
? This fundamentally important problem in image processing and computer vision has become particularly attractive as high definition displays dominate the market.
• The huge storage expense of UHD content and inspired by the aforementioned state-of-the-art super-resolution techniques, we propose a super-resolution generation solution in real-time with FPGA in this work.
• We propose a quantitative model for analysis and optimization to balance the utilization of limited hardware resources, the attainable frame rate, and the visual performance.
• Fixed-point precision is used, and a highly pipelined architecture is proposed for the real-time purpose.
• Though uniform memory partition strategies are explored in recent publications, e.g.,we adopt the micro-architecture proposed by to decouple the stencil access pattern from the computation.
? The first is a low-frequency interpolation stage, where bicubic interpolation is used for reconstructing the low-frequency parts of HR images.
? While one line buffer is used for memory write by the input low resolution image, the other four line buffers are used for memory read to the bicubic kernel.
? However, most of these CPU-based methods are far from reaching ideal performance as well as energy efficiency.
? FPGA-based accelerators for neural networks are gaining popularity because of its higher energy-efficiency comparing to GPUs and shorter development cycles comparing to ASICs.
|