Current Projects

Multi-frame Resolution Enhancement Methods Key Words: Multi-frame reconstruction, DCT domain processing, compressed-domain reconstruction

Project Members

• Ibrahim E. Pekkucuksen
• Michael J. Santoro

Project Description

It has been known for almost two decades by now that the sub-pixel shifts between subsequent video frames provide an excellent way of resolution enhancement provided that the sub-pixel motion vectors can be estimated with high enough accuracy. The recently proposed “super-resolution” reconstruction methods are all based on this idea. Once the imaging model, which produces the low-resolution observations from the high resolution target image, has been identified, one can formulate the resolution enhancement problem as an inverse problem and propose a solution strategy. But all these solution strategies are based on the fundamental assumption that we can achieve high accuracy sub-pixel motion detection using noisy low-resolution observations. It is a very well-known fact that if this assumption is not satisfied, all super-resolution methods are bound to fail. There exist two possible remedies to this problem. One can try to optimize the accuracy of the sub-pixel motion vectors and the reconstructed high resolution frame simultaneously in an iterative fashion, hoping that through the use of the reconstructed frame one can achieve higher accuracy in sub-pixel motion detection. This approach however is very demanding in terms of computational complexity. As a simpler alternative, one can try to detect the regions for which the accuracy of the sub-pixel motion vectors is not suitable for super-resolution reconstruction, and switch to some other reconstruction method for these regions, avoiding drastic errors. Our goal is to come up with robust methods to identify such regions with the ultimate goal of achieving robust super-resolution reconstruction.

Related Publications

• B. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Super-resolution reconstruction of compressed video using transform-domain statistics,” IEEE Transactions on Image Processing, vol. 13, no. 1, pp. 33- 43, January 2004 [pdf]
• B. Gunturk, Y. Altunbasak, and R. M. Mersereau, “Multiframe resolution-enhancement methods for compressed video,” IEEE Signal Processing Letters, vol. 9, no. 6, pp. 170-174, June 2002 [pdf]
• B. Gunturk, Y. Altunbasak, and R. M. Mersereau, “A multi-frame blocking artifact reduction method for transform coded video,” IEEE Transactions on Circuits and Systems for Video Technology, vol. 12, no. 4, pp. 269-276, April 2002 [pdf]
• B. Gunturk, Y. Altunbasak, and R. M. Mersereau, “A multi-frame blocking artifact reduction method for transform coded video,” IEEE Int. Conf. Acoust. Speech Sign. Proc., vol. 3, pp. 1789-1792, Salt Lake City, UT, May 2001 [pdf]
• B. Gunturk, Y. Altunbasak, and R. M. Mersereau, “A bayesian resolution enhancement framework for transform coded video,” Proc. IEEE Int. Conf. Image Proc., vol. 2, pp. 41-44, Thessaloniki, Greece, Oct. 2001 [pdf]