All of us face the problem of bad image cropping or resizing frequently, where either the picture details get blurred or its proportions get messed up (elongation or squeezing). Now, if this image needs to be used for an important situation, say in a legal document, it becomes extremely important to preserve its salient content and geometric consistency.

Researchers at IITGN has proposed a method* which prevents deformation of the important features of an image on different screens – an essential problem for 3D visualisation now-a-days. The comparison of this procedure with existing state-of-the-art image-retargeting techniques depicts that it is able to preserve the crucial details of a picture more efficiently. This implies that we would get a blur-free and proportionate image!

Dr Shanmuganathan Raman is a researcher and professor in Electrical Engineering at the institute. He, along with his PhD student Diptiben Patel, has developed a content-aware (related to artificial intelligence) image-retargeting technique which protects the important contextual information of an image and performs much better as compared to the methods which are used by the world in the present times.

“We wanted to resize the image content according to different display sizes, ranging from as small as an iPhone screen to as big as that of a TV above 100 inches. It is a multimedia (computational) problem — it’s about how you adapt a multimedia content to different target display devices, with a variety of aspect ratios and screen resolutions as well as printers. We wanted to re-scale the picture properly, but we did not want to compromise on its clear visibility,” said Professor Raman.

In simple terms, this mechanism is based on preserving only the essential information and throwing away details which are not important. The unimportant content is generally not perceptible to the human eye and discarding it would lead to effective rescaling of the image.

Seam carving was the first discrete method based on content-aware image retargeting. It can be understood as something which generates horizontal or vertical curves (seams). In order to resize a picture, it tracks down a curve with minimum energy level (least important information) and removes it. For this, there is a need to train the algorithm (a step-by-step procedure developed to solve a logical problem) involved in this process, about the difference between important and unimportant details.
“Our study aimed at developing improved algorithms which can differentiate between the essential and non-essential part of the image content more effectively. We also tried to remove multiple insignificant curves at the same time, called accelerated seam carving, as compared to the conventional system of deleting one seam at a time. Moreover, this research is the first of its kind to focus on safeguarding reflection scene symmetry and scene text,” he continued.

In case of different objects being present in the same image, this method resizes those objects simultaneously while successfully preserving the intrinsic geometry between them — the different depth levels of these objects with respect to one another remain protected, promoting better 3D visualisation.

Explaining further, Professor Raman said, “For example, the Taj Mahal at Agra is an intrinsic symmetry building. The existing methods of image retargeting destroy its picture symmetry in some way or the other (compressing or unnecessary cropping), while adjusting it to fit to different display sizes. But, our method tries to preserve the Taj as much as possible, which is an example of its efficient object-awareness. Another example is that of the presence of texts in an image. The traditional techniques often fail to protect the text and lead to collapsed and squeezed words which are not visible clearly. Our method overcomes this limitation, showing effective text-awareness quality.”

As of now, this approach has only been tested on images. It needs to be tested on videos to have an understanding of its real-time efficacy. In order to make the entire process completely seamless, there is a need of a more robust and real-time algorithm. It needs to be accelerated enough so as to minimise the time required for completing the entire procedure. For this, some of the operations need to be parallelised and solutions regarding the same need to found out. All this would, eventually, pave the way for the commercial utilisation of this method.

“You don’t take a photograph, you make it.” Ansel Easton Adams, renowned landscape photographer and environmentalist of the 20th century

— — — — — — — — — — — — — — — — — — — — — — — — — —— — — — — — — — — — — — — — — — — — — — — — — — — —— — — — — 

* The results of this research have been published in –

a) “Object occlusion guided stereo image retargeting”, Pattern Recognition Letters, DOI: 10.1016/j.patrec.2019.07.018, Jul. 2019.

b) “Reflection symmetry aware image retargeting”, Pattern Recognition Letters, DOI: 10.1016/j.patrec.2019.04.013, Jul. 2019.

** This story has also been published on Medium.

APEKSHA SRIVASTAVA

APEKSHA SRIVASTAVA

Senior Project Associate

An avid writer by passion and a researcher by education, Apeksha Srivastava works as a Senior Project Associate in External Communications at IITGN. She has pursued her MTech (2016-18) in Biological Engineering from IITGN, during which she was an Institute Gold Medalist. She was also an Institute Silver Medalist at Amity University, Lucknow, from where she completed her BTech in Biotechnology.