Improved content aware scene retargeting for retinitis pigmentosa patients
Identifieur interne : 000146 ( Pmc/Checkpoint ); précédent : 000145; suivant : 000147Improved content aware scene retargeting for retinitis pigmentosa patients
Auteurs : Walid I. Al-Atabany [Royaume-Uni, Égypte] ; Tzyy Tong [Royaume-Uni] ; Patrick A. Degenaar [Royaume-Uni]Source :
- BioMedical Engineering OnLine [ 1475-925X ] ; 2010.
Abstract
In this paper we present a novel scene retargeting technique to reduce the visual scene while maintaining the size of the key features. The algorithm is scalable to implementation onto portable devices, and thus, has potential for augmented reality systems to provide visual support for those with tunnel vision. We therefore test the efficacy of our algorithm on shrinking the visual scene into the remaining field of view for those patients.
Simple spatial compression of visual scenes makes objects appear further away. We have therefore developed an algorithm which removes low importance information, maintaining the size of the significant features. Previous approaches in this field have included
We have evaluated and compared our algorithm to the
Our approach allows us to perform content aware video resizing in real time using only information from previous frames to avoid jitter. Also our method has a great benefit over the ordinary resizing method and even over other image retargeting methods. We show that the benefit derived from this algorithm is significant to patients with fields of view 20° or less.
Url:
DOI: 10.1186/1475-925X-9-52
PubMed: 20846440
PubMed Central: 2949883
Affiliations:
Links toward previous steps (curation, corpus...)
Links to Exploration step
PMC:2949883Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Improved content aware scene retargeting for retinitis pigmentosa patients</title>
<author><name sortKey="Al Atabany, Walid I" sort="Al Atabany, Walid I" uniqKey="Al Atabany W" first="Walid I" last="Al-Atabany">Walid I. Al-Atabany</name>
<affiliation wicri:level="3"><nlm:aff id="I1">Department of Bioengineering, Imperial College London, London, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>Department of Bioengineering, Imperial College London, London</wicri:regionArea>
<placeName><settlement type="city">Londres</settlement>
<region type="country">Angleterre</region>
<region type="région" nuts="1">Grand Londres</region>
</placeName>
</affiliation>
<affiliation wicri:level="1"><nlm:aff id="I4">Department of Biomedical Engineering, Helwan University, Egypt</nlm:aff>
<country xml:lang="fr">Égypte</country>
<wicri:regionArea>Department of Biomedical Engineering, Helwan University</wicri:regionArea>
<wicri:noRegion>Helwan University</wicri:noRegion>
</affiliation>
</author>
<author><name sortKey="Tong, Tzyy" sort="Tong, Tzyy" uniqKey="Tong T" first="Tzyy" last="Tong">Tzyy Tong</name>
<affiliation wicri:level="3"><nlm:aff id="I2">Department of Computing, Imperial College London, London, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>Department of Computing, Imperial College London, London</wicri:regionArea>
<placeName><settlement type="city">Londres</settlement>
<region type="country">Angleterre</region>
<region type="région" nuts="1">Grand Londres</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Degenaar, Patrick A" sort="Degenaar, Patrick A" uniqKey="Degenaar P" first="Patrick A" last="Degenaar">Patrick A. Degenaar</name>
<affiliation wicri:level="1"><nlm:aff id="I3">School of Electrical, Electronic and Computer Engineering, Newcastle University, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>School of Electrical, Electronic and Computer Engineering, Newcastle University</wicri:regionArea>
<wicri:noRegion>Newcastle University</wicri:noRegion>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PMC</idno>
<idno type="pmid">20846440</idno>
<idno type="pmc">2949883</idno>
<idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2949883</idno>
<idno type="RBID">PMC:2949883</idno>
<idno type="doi">10.1186/1475-925X-9-52</idno>
<date when="2010">2010</date>
<idno type="wicri:Area/Pmc/Corpus">000158</idno>
<idno type="wicri:Area/Pmc/Curation">000158</idno>
<idno type="wicri:Area/Pmc/Checkpoint">000146</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Improved content aware scene retargeting for retinitis pigmentosa patients</title>
<author><name sortKey="Al Atabany, Walid I" sort="Al Atabany, Walid I" uniqKey="Al Atabany W" first="Walid I" last="Al-Atabany">Walid I. Al-Atabany</name>
<affiliation wicri:level="3"><nlm:aff id="I1">Department of Bioengineering, Imperial College London, London, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>Department of Bioengineering, Imperial College London, London</wicri:regionArea>
<placeName><settlement type="city">Londres</settlement>
<region type="country">Angleterre</region>
<region type="région" nuts="1">Grand Londres</region>
</placeName>
</affiliation>
<affiliation wicri:level="1"><nlm:aff id="I4">Department of Biomedical Engineering, Helwan University, Egypt</nlm:aff>
<country xml:lang="fr">Égypte</country>
<wicri:regionArea>Department of Biomedical Engineering, Helwan University</wicri:regionArea>
<wicri:noRegion>Helwan University</wicri:noRegion>
</affiliation>
</author>
<author><name sortKey="Tong, Tzyy" sort="Tong, Tzyy" uniqKey="Tong T" first="Tzyy" last="Tong">Tzyy Tong</name>
<affiliation wicri:level="3"><nlm:aff id="I2">Department of Computing, Imperial College London, London, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>Department of Computing, Imperial College London, London</wicri:regionArea>
<placeName><settlement type="city">Londres</settlement>
<region type="country">Angleterre</region>
<region type="région" nuts="1">Grand Londres</region>
</placeName>
</affiliation>
</author>
<author><name sortKey="Degenaar, Patrick A" sort="Degenaar, Patrick A" uniqKey="Degenaar P" first="Patrick A" last="Degenaar">Patrick A. Degenaar</name>
<affiliation wicri:level="1"><nlm:aff id="I3">School of Electrical, Electronic and Computer Engineering, Newcastle University, UK</nlm:aff>
<country xml:lang="fr">Royaume-Uni</country>
<wicri:regionArea>School of Electrical, Electronic and Computer Engineering, Newcastle University</wicri:regionArea>
<wicri:noRegion>Newcastle University</wicri:noRegion>
</affiliation>
</author>
</analytic>
<series><title level="j">BioMedical Engineering OnLine</title>
<idno type="eISSN">1475-925X</idno>
<imprint><date when="2010">2010</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass></textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en"><sec><title>Background</title>
<p>In this paper we present a novel scene retargeting technique to reduce the visual scene while maintaining the size of the key features. The algorithm is scalable to implementation onto portable devices, and thus, has potential for augmented reality systems to provide visual support for those with tunnel vision. We therefore test the efficacy of our algorithm on shrinking the visual scene into the remaining field of view for those patients.</p>
</sec>
<sec><title>Methods</title>
<p>Simple spatial compression of visual scenes makes objects appear further away. We have therefore developed an algorithm which removes low importance information, maintaining the size of the significant features. Previous approaches in this field have included <italic>seam carving</italic>
, which removes low importance seams from the scene, and <italic>shrinkability </italic>
which dynamically shrinks the scene according to a generated importance map. The former method causes significant artifacts and the latter is inefficient. In this work we have developed a new algorithm, combining the best aspects of both these two previous methods. In particular, our approach is to generate a <italic>shrinkability </italic>
importance map using as seam based approach. We then use it to dynamically shrink the scene in similar fashion to the <italic>shrinkability </italic>
method. Importantly, we have implemented it so that it can be used in real time without prior knowledge of future frames.</p>
</sec>
<sec><title>Results</title>
<p>We have evaluated and compared our algorithm to the <italic>seam carving </italic>
and image <italic>shrinkability </italic>
approaches from a content preservation perspective and a compression quality perspective. Also our technique has been evaluated and tested on a trial included 20 participants with simulated tunnel vision. Results show the robustness of our method at reducing scenes up to 50% with minimal distortion. We also demonstrate efficacy in its use for those with simulated tunnel vision of 22 degrees of field of view or less.</p>
</sec>
<sec><title>Conclusions</title>
<p>Our approach allows us to perform content aware video resizing in real time using only information from previous frames to avoid jitter. Also our method has a great benefit over the ordinary resizing method and even over other image retargeting methods. We show that the benefit derived from this algorithm is significant to patients with fields of view 20° or less.</p>
</sec>
</div>
</front>
<back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Foster, A" uniqKey="Foster A">A Foster</name>
</author>
<author><name sortKey="Resnikoff, S" uniqKey="Resnikoff S">S Resnikoff</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Hamel, C" uniqKey="Hamel C">C Hamel</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Vargas Martin, F" uniqKey="Vargas Martin F">F Vargas-Martín</name>
</author>
<author><name sortKey="Pelaez Coca, Md" uniqKey="Pelaez Coca M">MD Peláez-Coca</name>
</author>
<author><name sortKey="Ros, E" uniqKey="Ros E">E Ros</name>
</author>
<author><name sortKey="Diaz, J" uniqKey="Diaz J">J Diaz</name>
</author>
<author><name sortKey="Mota, S" uniqKey="Mota S">S Mota</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Fullerton, M" uniqKey="Fullerton M">M Fullerton</name>
</author>
<author><name sortKey="Peli, E" uniqKey="Peli E">E Peli</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wolffsohn, Js" uniqKey="Wolffsohn J">JS Wolffsohn</name>
</author>
<author><name sortKey="Mukhopadhyay, D" uniqKey="Mukhopadhyay D">D Mukhopadhyay</name>
</author>
<author><name sortKey="Rubinstein, M" uniqKey="Rubinstein M">M Rubinstein</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Atabany, W" uniqKey="Atabany W">W Atabany</name>
</author>
<author><name sortKey="Degenaar, P" uniqKey="Degenaar P">P Degenaar</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bennett, J" uniqKey="Bennett J">J Bennett</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Peli, E" uniqKey="Peli E">E Peli</name>
</author>
<author><name sortKey="Luo, G" uniqKey="Luo G">G Luo</name>
</author>
<author><name sortKey="Bowers, A" uniqKey="Bowers A">A Bowers</name>
</author>
<author><name sortKey="Rensing, N" uniqKey="Rensing N">N Rensing</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Simons, Dj" uniqKey="Simons D">DJ Simons</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Suh, B" uniqKey="Suh B">B Suh</name>
</author>
<author><name sortKey="Ling, H" uniqKey="Ling H">H Ling</name>
</author>
<author><name sortKey="Bederson, Bb" uniqKey="Bederson B">BB Bederson</name>
</author>
<author><name sortKey="Jacobs, Dw" uniqKey="Jacobs D">DW Jacobs</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Chen, Lq" uniqKey="Chen L">LQ Chen</name>
</author>
<author><name sortKey="Xie, X" uniqKey="Xie X">X Xie</name>
</author>
<author><name sortKey="Fan, X" uniqKey="Fan X">X Fan</name>
</author>
<author><name sortKey="Ma, Wy" uniqKey="Ma W">WY Ma</name>
</author>
<author><name sortKey="Zhang, Hj" uniqKey="Zhang H">HJ Zhang</name>
</author>
<author><name sortKey="Zhou, Hq" uniqKey="Zhou H">HQ Zhou</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Liu, F" uniqKey="Liu F">F Liu</name>
</author>
<author><name sortKey="Gleicher, M" uniqKey="Gleicher M">M Gleicher</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Setlur, V" uniqKey="Setlur V">V Setlur</name>
</author>
<author><name sortKey="Takagi, S" uniqKey="Takagi S">S Takagi</name>
</author>
<author><name sortKey="Raskar, R" uniqKey="Raskar R">R Raskar</name>
</author>
<author><name sortKey="Gleicher, M" uniqKey="Gleicher M">M Gleicher</name>
</author>
<author><name sortKey="Gooch, B" uniqKey="Gooch B">B Gooch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Setlur, V" uniqKey="Setlur V">V Setlur</name>
</author>
<author><name sortKey="Lechner, T" uniqKey="Lechner T">T Lechner</name>
</author>
<author><name sortKey="Nienhaus, M" uniqKey="Nienhaus M">M Nienhaus</name>
</author>
<author><name sortKey="Gooch, B" uniqKey="Gooch B">B Gooch</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Avidan, S" uniqKey="Avidan S">S Avidan</name>
</author>
<author><name sortKey="Shamir, A" uniqKey="Shamir A">A Shamir</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Rubinstein, M" uniqKey="Rubinstein M">M Rubinstein</name>
</author>
<author><name sortKey="Shamir, A" uniqKey="Shamir A">A Shamir</name>
</author>
<author><name sortKey="Avidan, S" uniqKey="Avidan S">S Avidan</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Wolf, L" uniqKey="Wolf L">L Wolf</name>
</author>
<author><name sortKey="Guttmann, M" uniqKey="Guttmann M">M Guttmann</name>
</author>
<author><name sortKey="Cohen Or, D" uniqKey="Cohen Or D">D Cohen-Or</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Zhang, Yf" uniqKey="Zhang Y">YF Zhang</name>
</author>
<author><name sortKey="Hu, Sm" uniqKey="Hu S">SM Hu</name>
</author>
<author><name sortKey="Martin, Rr" uniqKey="Martin R">RR Martin</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Kim, J S" uniqKey="Kim J">J-S Kim</name>
</author>
<author><name sortKey="Kim, J H" uniqKey="Kim J">J-H Kim</name>
</author>
<author><name sortKey="Kim, C S" uniqKey="Kim C">C-S Kim</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Fleck, Mm" uniqKey="Fleck M">MM Fleck</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Canny, J" uniqKey="Canny J">J Canny</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Jobson, Dj" uniqKey="Jobson D">DJ Jobson</name>
</author>
<author><name sortKey="Rahman, Z" uniqKey="Rahman Z">Z Rahman</name>
</author>
<author><name sortKey="Woodell, Ga" uniqKey="Woodell G">GA Woodell</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Liu, Sc" uniqKey="Liu S">SC Liu</name>
</author>
<author><name sortKey="Fu, Cw" uniqKey="Fu C">CW Fu</name>
</author>
<author><name sortKey="Chang, Sy" uniqKey="Chang S">SY Chang</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Bovik, Ac" uniqKey="Bovik A">AC Bovik</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Fant, Km" uniqKey="Fant K">KM Fant</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Spitz, Al" uniqKey="Spitz A">AL Spitz</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Degenaar, P" uniqKey="Degenaar P">P Degenaar</name>
</author>
<author><name sortKey="Grossman, N" uniqKey="Grossman N">N Grossman</name>
</author>
<author><name sortKey="Memon, Ma" uniqKey="Memon M">MA Memon</name>
</author>
<author><name sortKey="Burrone, J" uniqKey="Burrone J">J Burrone</name>
</author>
<author><name sortKey="Dawson, M" uniqKey="Dawson M">M Dawson</name>
</author>
<author><name sortKey="Drakakis, E" uniqKey="Drakakis E">E Drakakis</name>
</author>
<author><name sortKey="Neil, M" uniqKey="Neil M">M Neil</name>
</author>
<author><name sortKey="Nikolic, K" uniqKey="Nikolic K">K Nikolic</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Al Atabany, W" uniqKey="Al Atabany W">W Al-Atabany</name>
</author>
<author><name sortKey="Memon, M" uniqKey="Memon M">M Memon</name>
</author>
<author><name sortKey="Downes, S" uniqKey="Downes S">S Downes</name>
</author>
<author><name sortKey="Degenaar, P" uniqKey="Degenaar P">P Degenaar</name>
</author>
</analytic>
</biblStruct>
<biblStruct><analytic><author><name sortKey="Atabany, W" uniqKey="Atabany W">W Atabany</name>
</author>
<author><name sortKey="Degenaar, P" uniqKey="Degenaar P">P Degenaar</name>
</author>
</analytic>
</biblStruct>
</listBibl>
</div1>
</back>
</TEI>
<pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Biomed Eng Online</journal-id>
<journal-title-group><journal-title>BioMedical Engineering OnLine</journal-title>
</journal-title-group>
<issn pub-type="epub">1475-925X</issn>
<publisher><publisher-name>BioMed Central</publisher-name>
</publisher>
</journal-meta>
<article-meta><article-id pub-id-type="pmid">20846440</article-id>
<article-id pub-id-type="pmc">2949883</article-id>
<article-id pub-id-type="publisher-id">1475-925X-9-52</article-id>
<article-id pub-id-type="doi">10.1186/1475-925X-9-52</article-id>
<article-categories><subj-group subj-group-type="heading"><subject>Research</subject>
</subj-group>
</article-categories>
<title-group><article-title>Improved content aware scene retargeting for retinitis pigmentosa patients</article-title>
</title-group>
<contrib-group><contrib contrib-type="author" corresp="yes" id="A1"><name><surname>Al-Atabany</surname>
<given-names>Walid I</given-names>
</name>
<xref ref-type="aff" rid="I1">1</xref>
<xref ref-type="aff" rid="I4">4</xref>
<email>walid.atbany06@imperial.ac.uk</email>
</contrib>
<contrib contrib-type="author" id="A2"><name><surname>Tong</surname>
<given-names>Tzyy</given-names>
</name>
<xref ref-type="aff" rid="I2">2</xref>
<email>tzyy.tong04@imperial.ac.uk</email>
</contrib>
<contrib contrib-type="author" id="A3"><name><surname>Degenaar</surname>
<given-names>Patrick A</given-names>
</name>
<xref ref-type="aff" rid="I3">3</xref>
<email>Patrick.Degenaar@newcastle.ac.uk</email>
</contrib>
</contrib-group>
<aff id="I1"><label>1</label>
Department of Bioengineering, Imperial College London, London, UK</aff>
<aff id="I2"><label>2</label>
Department of Computing, Imperial College London, London, UK</aff>
<aff id="I3"><label>3</label>
School of Electrical, Electronic and Computer Engineering, Newcastle University, UK</aff>
<aff id="I4"><label>4</label>
Department of Biomedical Engineering, Helwan University, Egypt</aff>
<pub-date pub-type="collection"><year>2010</year>
</pub-date>
<pub-date pub-type="epub"><day>16</day>
<month>9</month>
<year>2010</year>
</pub-date>
<volume>9</volume>
<fpage>52</fpage>
<lpage>52</lpage>
<history><date date-type="received"><day>16</day>
<month>4</month>
<year>2010</year>
</date>
<date date-type="accepted"><day>16</day>
<month>9</month>
<year>2010</year>
</date>
</history>
<permissions><copyright-statement>Copyright ©2010 Al-Atabany et al; licensee BioMed Central Ltd.</copyright-statement>
<copyright-year>2010</copyright-year>
<copyright-holder>Al-Atabany et al; licensee BioMed Central Ltd.</copyright-holder>
<license license-type="open-access" xlink:href="http://creativecommons.org/licenses/by/2.0"><license-p>This is an Open Access article distributed under the terms of the Creative Commons Attribution License (<ext-link ext-link-type="uri" xlink:href="http://creativecommons.org/licenses/by/2.0">http://creativecommons.org/licenses/by/2.0</ext-link>
), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.</license-p>
</license>
</permissions>
<self-uri xlink:href="http://www.biomedical-engineering-online.com/content/9/1/52"></self-uri>
<abstract><sec><title>Background</title>
<p>In this paper we present a novel scene retargeting technique to reduce the visual scene while maintaining the size of the key features. The algorithm is scalable to implementation onto portable devices, and thus, has potential for augmented reality systems to provide visual support for those with tunnel vision. We therefore test the efficacy of our algorithm on shrinking the visual scene into the remaining field of view for those patients.</p>
</sec>
<sec><title>Methods</title>
<p>Simple spatial compression of visual scenes makes objects appear further away. We have therefore developed an algorithm which removes low importance information, maintaining the size of the significant features. Previous approaches in this field have included <italic>seam carving</italic>
, which removes low importance seams from the scene, and <italic>shrinkability </italic>
which dynamically shrinks the scene according to a generated importance map. The former method causes significant artifacts and the latter is inefficient. In this work we have developed a new algorithm, combining the best aspects of both these two previous methods. In particular, our approach is to generate a <italic>shrinkability </italic>
importance map using as seam based approach. We then use it to dynamically shrink the scene in similar fashion to the <italic>shrinkability </italic>
method. Importantly, we have implemented it so that it can be used in real time without prior knowledge of future frames.</p>
</sec>
<sec><title>Results</title>
<p>We have evaluated and compared our algorithm to the <italic>seam carving </italic>
and image <italic>shrinkability </italic>
approaches from a content preservation perspective and a compression quality perspective. Also our technique has been evaluated and tested on a trial included 20 participants with simulated tunnel vision. Results show the robustness of our method at reducing scenes up to 50% with minimal distortion. We also demonstrate efficacy in its use for those with simulated tunnel vision of 22 degrees of field of view or less.</p>
</sec>
<sec><title>Conclusions</title>
<p>Our approach allows us to perform content aware video resizing in real time using only information from previous frames to avoid jitter. Also our method has a great benefit over the ordinary resizing method and even over other image retargeting methods. We show that the benefit derived from this algorithm is significant to patients with fields of view 20° or less.</p>
</sec>
</abstract>
</article-meta>
</front>
<body><sec><title>Background</title>
<p>There are thought to be 38 million people suffering from blindness worldwide, and this number is expected to double over the next 25 years [<xref ref-type="bibr" rid="B1">1</xref>
]. Additionally, there are more than 124 million people who have severely impaired vision. The low vision pathologies of this latter group can be divided mainly into two categories; those that predominantly suffer from a loss of visual acuity such as Macular Degeneration (MD), and those that predominantly suffer from a reduction in the overall visual field, such as Retinitis Pigmentosa (RP). RP in particular (population prevalence ~1:4000 [<xref ref-type="bibr" rid="B2">2</xref>
]) causes a tunnel vision with decreasing peripheral fields as the condition progresses.</p>
<p>For those with central visual impairment, conventional low vision aids (LVAs) can provide magnification in order to compensate for reduced visual acuity. Also, electronically enhanced visual aids have been proposed which offer a number of distinct advantages over conventional LVAs by enhancing the contrast without the need of image magnification [<xref ref-type="bibr" rid="B3">3</xref>
-<xref ref-type="bibr" rid="B6">6</xref>
].</p>
<p>Severe visual field (VF) impairment (those with a 20° in remaining tunnel or worse) can greatly affect a patient's mobility and navigation. Despite ongoing research into genetic and pharmacological therapies [<xref ref-type="bibr" rid="B7">7</xref>
], there is currently no effective treatment for RP patients which can significantly slow or arrest the disease. Traditional low-vision aids for these patients have included de-magnifying optics to expand the remaining visual field of those patients. However, such demagnification comes at the cost of optical (fish-eye) distortion and a loss of resolution (i.e. the objects seem more distant).</p>
<p>Recently, Peli et. al. developed an augmented vision system [<xref ref-type="bibr" rid="B8">8</xref>
] which multiplexing minified edges over the original scene on a see-through display. However, there is the potential for inattentional blindness, which is the inability of observers to maintain awareness of events in more than one of two superimposed scenes [<xref ref-type="bibr" rid="B9">9</xref>
].</p>
<p>This paper introduces a new method for image retargeting for those with peripheral vision impairment without degrading the resolution or adding more complexities to the visual scene.</p>
<p>Image resizing is an interesting topic in the image processing field, due to the increasing demand for displaying images and videos on a variety of display devices of different resolutions or aspect ratios. Standard image resizing techniques, such as scaling and cropping, are not efficient. Scaling is applied uniformly by reducing the sampling over the whole image. As with its optical (demagnifying) counterpart, it results in the key features becoming smaller and appearing further away. An alternative approach is to use scene cropping, as performed by Suh et al. [<xref ref-type="bibr" rid="B10">10</xref>
] and Chen et. al. [<xref ref-type="bibr" rid="B11">11</xref>
], which involves finding the best rectangular sub-window in the image to be cropped. This is useful only if there is a single important feature in the image then the image can be cropped and scaled to fit. However, Images with multiple important features present a more challenging case for image cropping.</p>
<p>Recently, important progress has been achieved in the development of content-aware image and video resizing techniques. Liu and Gleicher [<xref ref-type="bibr" rid="B12">12</xref>
] proposed a different image retargeting algorithm, which determines a region of interest (ROI) and then applies a novel fisheye-view warping that applies a piecewise linear scaling function in each dimension to the image to achieve a target image size. Their algorithm is simple, but the warping may cause distortions that look unnatural.</p>
<p>Setlur et al. [<xref ref-type="bibr" rid="B13">13</xref>
,<xref ref-type="bibr" rid="B14">14</xref>
] proposed an alternative approach for retargeting large images to small size displays by segmenting the proposed image into a background layer and different ROIs objects, then cuts the ROIs from the image, and fills the holes with an inpainting scheme. Then it rescales the background image and finally pastes the ROIs back to the image. Despite the quality of the resized image, this approach relies strongly on the quality of segmentation which can be difficult, requiring complex feature recognition to be able to be performed properly.</p>
<p>Avidan and Shamir [<xref ref-type="bibr" rid="B15">15</xref>
] recently provided a new algorithm called <italic>seam carving</italic>
. This algorithm alters the dimensions of an image by removing a connected path of pixels, called a seam, from an input image repeatedly to achieve a target size. Although the technique shows good results for static images, it has limitations. When an image is overly compressed, the algorithm starts to carve out important objects, yielding unnatural artefacts.</p>
<p>Extending <italic>seam carving </italic>
to video, by treating each frame as an image and resizing it independently, creates jittery artefacts due to the lack of temporal coherency. Rubinstein et al. [<xref ref-type="bibr" rid="B16">16</xref>
] improved the algorithm by treating video as a 3D cube and extending <italic>seam carving </italic>
from 1-D paths to 2-D manifolds in a 3-D volume. There are two limitations; Firstly, if there are moving objects which travel from top to down and from left to right, then it will be so difficult to find a 2-D plane that avoid crossing these objects causing unnatural artefacts. Secondly, this algorithm cannot process real time video, and is therefore not useful for those with restricted field of view.</p>
<p>Wolf et al. [<xref ref-type="bibr" rid="B17">17</xref>
] proposed a system to retarget video by using non-uniform global warping. Given an input image, their algorithm first computes the importance of each pixel, based on spatial edges, face detection and motion detection. Then, based on the <italic>importance </italic>
map, it forms a system of linear equations solved by a least squares manner.</p>
<p>A similar system was later proposed by Zhang et al. [<xref ref-type="bibr" rid="B18">18</xref>
], where they used the same <italic>importance </italic>
map of the Wolf method, but they then used it to calculate a shrinkability matrix. Each pixel of the image is compressed according to it is shrinkage value. This algorithm shows fewer artefacts than the <italic>seam carving </italic>
method because it shrinks the image rather than removing pixels. However, the compression efficiency of this method in keeping the original size of the important objects while minimizing less important features is low.</p>
<p>Kim et al. [<xref ref-type="bibr" rid="B19">19</xref>
] recently provided a strip based image and video retargeting technique. Their approach divides the image into strips and compresses each strip individually based on the gradient complexity within each strip. The advantage with this approach is that it can more uniformly distribute the compression across the image. However, this technique can distort the objects in the image. This is because part of one object may be scaled with a scaling factor different to the other parts when an object resides between two strips. Furthermore, this algorithm is not efficient in video as moving objects can cross different stripe boundaries causing significant artefacts in the resultant target video.</p>
<p>From a comprehensive review of the literature it is clear that there are some trade-offs in these prior methods. In this paper we combine the advantages of the <italic>seam carving </italic>
and the pixel <italic>shrinkability </italic>
method, by designing a novel <italic>importance </italic>
map based on the <italic>seam carving </italic>
which we use to shrink the pixels. Also, we show how our method can be scalable to perform content aware video resizing in real time. This is possible as our method only needs to consider previous rather than future frames.</p>
</sec>
<sec sec-type="methods"><title>Methods</title>
<p>Our system, described in Figure <xref ref-type="fig" rid="F1">1</xref>
, consists of five main steps;</p>
<fig id="F1" position="float"><label>Figure 1</label>
<caption><p><bold>Algorithm flow</bold>
. This diagram illustrates the major steps in our algorithm, the top raw shows the processing in the horizontal direction and the bottom raw shows the vertical one.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-1"></graphic>
</fig>
<p>1. A per-pixel importance matrix is computed indicating the significance of each pixel. This is a combination of two measures: a local saliency gradient map, and a block difference motion detector map.</p>
<p>2. A modified importance map is computed based on the seam pixel locations.</p>
<p>3. A shrinkage map is computed from the modified importance map.</p>
<p>4. This shrinking map is scaled to resize the whole image to the desired k columns.</p>
<p>5. Finally, a remapping algorithm is applied to re-map each pixel in the original image into its new location in the retargeted image. Thus, a pixel with a low shrinking value is mapped to a distance of approximately one from its left neighbor, while a pixel of high shrinking value is mapped closer to its neighbor.</p>
<p>These five steps are repeated for the vertical resizing.</p>
<sec><title>A) Generating the importance matrix</title>
<p>Our method of generating an <italic>importance </italic>
matrix has similarities to that of Wolf [<xref ref-type="bibr" rid="B17">17</xref>
]. However, to calculate the saliency map, we use a pyramidal edge detection method to improve computational efficiency. Each pixel in the <italic>importance </italic>
matrix is a combination between the saliency of the current pixel in the source image and the dynamic information of this pixel compared to its location in the previous frame. Values range between 0 and 1, where 0 refers to non- important pixels and 1 refers to high importance pixels.</p>
<sec><title>1) Spatial saliency map</title>
<p>To obtain the spatial saliency map, we use an algorithm described previously by Fleck [<xref ref-type="bibr" rid="B20">20</xref>
] which based on a modified Canny filter [<xref ref-type="bibr" rid="B21">21</xref>
]. Briefly, simple masks [-1, 0, 1] are used to compute the first derivative in four directions: <italic>H </italic>
(horizontal), <italic>V </italic>
(vertical), <italic>D1</italic>
, and <italic>D2 </italic>
(diagonal). The <italic>X </italic>
and <italic>Y </italic>
spatial gradients are then computed by projecting the diagonal differences on both axes.</p>
<p><disp-formula id="bmcM1"><label>(1)</label>
<mml:math id="M1" name="1475-925X-9-52-i1" overflow="scroll"><mml:mrow><mml:mi>X</mml:mi>
<mml:mo>=</mml:mo>
<mml:mi>H</mml:mi>
<mml:mo>+</mml:mo>
<mml:mfrac><mml:mrow><mml:mi>D</mml:mi>
<mml:mn>1</mml:mn>
<mml:mo>+</mml:mo>
<mml:mi>D</mml:mi>
<mml:mn>2</mml:mn>
</mml:mrow>
<mml:mn>2</mml:mn>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p><disp-formula id="bmcM2"><label>(2)</label>
<mml:math id="M2" name="1475-925X-9-52-i2" overflow="scroll"><mml:mrow><mml:mi>Y</mml:mi>
<mml:mo>=</mml:mo>
<mml:mi>V</mml:mi>
<mml:mo>+</mml:mo>
<mml:mfrac><mml:mrow><mml:mi>D</mml:mi>
<mml:mn>1</mml:mn>
<mml:mo>−</mml:mo>
<mml:mi>D</mml:mi>
<mml:mn>2</mml:mn>
</mml:mrow>
<mml:mn>2</mml:mn>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>The amplitude of the spatial gradient is:</p>
<p><disp-formula id="bmcM3"><label>(3)</label>
<mml:math id="M3" name="1475-925X-9-52-i3" overflow="scroll"><mml:mrow><mml:mi>W</mml:mi>
<mml:mi>s</mml:mi>
<mml:mo>=</mml:mo>
<mml:msqrt><mml:mrow><mml:msup><mml:mi>X</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
<mml:mo>+</mml:mo>
<mml:msup><mml:mi>Y</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
</mml:mrow>
</mml:msqrt>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>As simple high frequency (small kernel) derivatives of this form can be lossy in their boundary detection, we therefore use a multi-scale pyramidal approach with three kernel sizes to obtain lower frequency (large kernel) spatial derivatives [<xref ref-type="bibr" rid="B22">22</xref>
].</p>
</sec>
<sec><title>2) Temporal saliency map</title>
<p>A Motion saliency map is a map used to identify moving objects. Because the human eye is very sensitive to motion, retargeting dynamic scenes while preserving the temporal context is very important. Motion is detected based on a method proposed by Liu et al. [<xref ref-type="bibr" rid="B23">23</xref>
], which is relatively easy to implement and requires low computational power. The image is divided into NxN blocks and motion in each block is calculated by taking the weighted average of intensity difference of each pixel, so that the motion map <italic>W<sub>T</sub>
</italic>
(<italic>x</italic>
, <italic>y</italic>
) is set to one if the block containing the pixel (<italic>x</italic>
, <italic>y</italic>
) has motion, and zero otherwise. We uses N = 4 in all the processed images and videos in this paper.</p>
<p>The combined spatio-temporal <italic>importance </italic>
matrix then is the combination of the spatial and temporal saliency maps, normalised to between 0 and 1.</p>
<p><disp-formula id="bmcM4"><label>(4)</label>
<mml:math id="M4" name="1475-925X-9-52-i4" overflow="scroll"><mml:mrow><mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mi>S</mml:mi>
</mml:msub>
<mml:mo>+</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mi>T</mml:mi>
</mml:msub>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p><disp-formula id="bmcM5"><label>(5)</label>
<mml:math id="M5" name="1475-925X-9-52-i5" overflow="scroll"><mml:mrow><mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mrow><mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo>−</mml:mo>
<mml:mi>min</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow><mml:mi>max</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mi>min</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
</sec>
</sec>
<sec><title>B) Modifying the importance matrix</title>
<p>Figure <xref ref-type="fig" rid="F2">2(b)</xref>
and <xref ref-type="fig" rid="F2">2(c)</xref>
, show the generated spatial and spatio-temporal <italic>importance </italic>
maps on a two-frame moving image. The arrows defining the space within the object and on the background represent areas of similar importance. Using the standard <italic>shrinkability </italic>
approach described above would mean that both regions would be equally compressed. To give higher importance values to the entire body of such foreground objects, we would have to segment the entire image to determine which regions correspond to important bodies. This would require object recognition algorithms which are difficult and complex, requiring high levels of computational effort. Instead, we can adapt the seam carving approach described by Avidan and Shamir [<xref ref-type="bibr" rid="B15">15</xref>
].</p>
<fig id="F2" position="float"><label>Figure 2</label>
<caption><p><bold>Basic algorithm outcomes</bold>
. The output from each stage in our Seam Shrinking algorithm, when applied to two frames of video file. (a) is the two input images, (b), (c), (d) the spatial, spatio-temporal and modified importance matrix, respectively, (e), (f) the shrinkage map when using the unmodified and modified importance matrix, respectively, (g), (h), and (i) the retargeted image when using just interpolation, unmodified and modified shrinkage map, respectively.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-2"></graphic>
</fig>
<p>In the <italic>seam carving </italic>
method, a cumulative energy map is generated based on the spatial <italic>importance </italic>
map similar to that described in equation 3. We modify this to include temporal saliency as given in equation 5. We then search for seams with lowest energy and then give them very low importance values.</p>
<p>In an <italic>n </italic>
× <italic>m </italic>
image I, a vertical seam is defined to be</p>
<p><disp-formula id="bmcM6"><label>(6)</label>
<mml:math id="M6" name="1475-925X-9-52-i6" overflow="scroll"><mml:mtable columnalign="left"><mml:mtr><mml:mtd><mml:msup><mml:mtext>S</mml:mtext>
<mml:mtext>X</mml:mtext>
</mml:msup>
<mml:mo>=</mml:mo>
<mml:msubsup><mml:mrow><mml:mo>{</mml:mo>
<mml:mrow><mml:msubsup><mml:mtext>S</mml:mtext>
<mml:mtext>i</mml:mtext>
<mml:mtext>x</mml:mtext>
</mml:msubsup>
</mml:mrow>
<mml:mo>}</mml:mo>
</mml:mrow>
<mml:mrow><mml:mtext>i</mml:mtext>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>n</mml:mi>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:msubsup><mml:mrow><mml:mo>{</mml:mo>
<mml:mrow><mml:mrow><mml:mo>(</mml:mo>
<mml:mrow><mml:mtext>x</mml:mtext>
<mml:mrow><mml:mo>(</mml:mo>
<mml:mtext>i</mml:mtext>
<mml:mo>)</mml:mo>
</mml:mrow>
<mml:mo>,</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mtext>i</mml:mtext>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
<mml:mo>}</mml:mo>
</mml:mrow>
<mml:mrow><mml:mtext>i</mml:mtext>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>n</mml:mi>
</mml:msubsup>
<mml:mo>,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mi>s</mml:mi>
<mml:mo>.</mml:mo>
<mml:mi>t</mml:mi>
<mml:mo>.</mml:mo>
<mml:mo>∀</mml:mo>
<mml:mtext>i</mml:mtext>
<mml:mo>,</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mo>|</mml:mo>
<mml:mtext>x</mml:mtext>
<mml:mo stretchy="false">(</mml:mo>
<mml:mtext>i</mml:mtext>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mtext>x</mml:mtext>
<mml:mo stretchy="false">(</mml:mo>
<mml:mtext>i</mml:mtext>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>|</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mo>≤</mml:mo>
<mml:mn>1</mml:mn>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:math>
</disp-formula>
</p>
<p>Where x(i) represents the column number for a given row, as the seam path should has only one pixel in each row of the image. This condition |x(i) − x(i − 1)| ≤ 1 is to make sure that the pixels along the seam path are connected.</p>
<p>Similarly, a horizontal seam is defined as follows:</p>
<p><disp-formula id="bmcM7"><label>(7)</label>
<mml:math id="M7" name="1475-925X-9-52-i7" overflow="scroll"><mml:mtable columnalign="left"><mml:mtr><mml:mtd><mml:msup><mml:mi>S</mml:mi>
<mml:mtext>Y</mml:mtext>
</mml:msup>
<mml:mo>=</mml:mo>
<mml:msubsup><mml:mrow><mml:mo>{</mml:mo>
<mml:mrow><mml:msubsup><mml:mi>S</mml:mi>
<mml:mi>j</mml:mi>
<mml:mi>y</mml:mi>
</mml:msubsup>
</mml:mrow>
<mml:mo>}</mml:mo>
</mml:mrow>
<mml:mrow><mml:mi>j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>m</mml:mi>
</mml:msubsup>
<mml:mo>=</mml:mo>
<mml:msubsup><mml:mrow><mml:mo>{</mml:mo>
<mml:mrow><mml:mrow><mml:mo>(</mml:mo>
<mml:mrow><mml:mi>j</mml:mi>
<mml:mo>,</mml:mo>
<mml:mtext>y</mml:mtext>
<mml:mrow><mml:mo>(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
<mml:mo>}</mml:mo>
</mml:mrow>
<mml:mrow><mml:mi>j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>m</mml:mi>
</mml:msubsup>
<mml:mo>,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mi>s</mml:mi>
<mml:mo>.</mml:mo>
<mml:mi>t</mml:mi>
<mml:mo>.</mml:mo>
<mml:mo>∀</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>,</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mo>|</mml:mo>
<mml:mi>y</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mi>y</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>|</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mo>≤</mml:mo>
<mml:mn>1</mml:mn>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:math>
</disp-formula>
</p>
<p>As with the Avidan and Shamir [<xref ref-type="bibr" rid="B15">15</xref>
] approach, we define the energy of a seam as the sum of all pixels' energy in this seam path. We then use dynamic programming to look for the seam with minimal energy. Taking the vertical seam as an example, we establish the cumulative energy matrix <italic>M</italic>
:</p>
<p><disp-formula id="bmcM8"><label>(8)</label>
<mml:math id="M8" name="1475-925X-9-52-i8" overflow="scroll"><mml:mtable columnalign="left"><mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mi>M</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>S</mml:mi>
<mml:mi>T</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mo>+</mml:mo>
<mml:mi>m</mml:mi>
<mml:mi>i</mml:mi>
<mml:mi>n</mml:mi>
<mml:mo>{</mml:mo>
<mml:mi>M</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>,</mml:mo>
<mml:mtext> </mml:mtext>
<mml:mi>M</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mi>M</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>}</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:math>
</disp-formula>
</p>
<p>Then starting from the last row in the <italic>M </italic>
matrix, we search for the minimal cumulative pixel. After that, we work backwards from this pixel to obtain an optimal vertical seam by finding the minimum of the three neighboring pixels of this pixel in the previous row and then save this pixel to the seam path. At this point, Avidan and Shamir [<xref ref-type="bibr" rid="B15">15</xref>
] would adjust the image width by removing this optimal vertical seam. These steps are repeated until the desired size is achieved.</p>
<p>At this point our method deviates from Avidan and Shamir algorithm in that we do not remove the seams; instead we save the locations of the pixels along the path of the all seams, and we generate another matrix <italic>U </italic>
of size N×K, as K is the number of columns by which the input image should be shrunk. The <italic>U </italic>
matrix contains the <bold><italic>x </italic>
</bold>
and <italic>y </italic>
positions of the K seams.</p>
<p><disp-formula id="bmcM9"><label>(9)</label>
<mml:math id="M9" name="1475-925X-9-52-i9" overflow="scroll"><mml:mrow><mml:mi>U</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:msup><mml:mi>S</mml:mi>
<mml:mi>X</mml:mi>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mtext>x</mml:mtext>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>By knowing the locations of lowest energy pixels, we rescale all the pixels along the path of all the seams to very low importance values as following:</p>
<p><disp-formula id="bmcM10"><label>(10)</label>
<mml:math id="M10" name="1475-925X-9-52-i10" overflow="scroll"><mml:mrow><mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>U</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mn>0.001</mml:mn>
<mml:mo>*</mml:mo>
<mml:mfrac><mml:mrow><mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>U</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mi>min</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>U</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
<mml:mrow><mml:mi>max</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>U</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mi>min</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>U</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>The next step after adapting the <italic>importance </italic>
matrix is to shrink the source image based on the importance of each pixel. However, if we simply shrink each row of the image independently, this would cause unnatural artefacts to the image content by creating a zigzag effect as shown in Figure <xref ref-type="fig" rid="F3">3(b)</xref>
. Thus, we have to ensure that <italic>W</italic>
(<italic>i</italic>
, <italic>j</italic>
), is equal to <italic>W</italic>
(<italic>i</italic>
, <italic>j </italic>
- 1), in case of horizontal retargeting, in order to preserve continuity between rows. To do that, we assign a fixed importance value for each column by applying a moving average window of size <italic>L </italic>
and overlapping by <italic>L</italic>
/2 on each column in the <italic>importance </italic>
matrix [<xref ref-type="bibr" rid="B24">24</xref>
], and then taking the maximum of these average values for each column as following.</p>
<fig id="F3" position="float"><label>Figure 3</label>
<caption><p><bold>The effect of discontinuity of rows</bold>
. (a) A 400 × 650 image. The image is retargeted to 400 × 400 by shrinking each row independently (b), causing a Zig-Zag, and when preserving row continuity (c).</p>
</caption>
<graphic xlink:href="1475-925X-9-52-3"></graphic>
</fig>
<p><disp-formula id="bmcM11"><label>(11)</label>
<mml:math id="M11" name="1475-925X-9-52-i11" overflow="scroll"><mml:mtable><mml:mtr><mml:mtd><mml:msub><mml:mover accent="true"><mml:mi>W</mml:mi>
<mml:mo>¯</mml:mo>
</mml:mover>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:msub><mml:mover accent="true"><mml:mi>W</mml:mi>
<mml:mo>¯</mml:mo>
</mml:mover>
<mml:mrow><mml:mi>r</mml:mi>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>+</mml:mo>
<mml:mstyle scriptlevel="+1"><mml:mfrac><mml:mn>1</mml:mn>
<mml:mi>L</mml:mi>
</mml:mfrac>
</mml:mstyle>
<mml:mo stretchy="false">[</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mo>−</mml:mo>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>r</mml:mi>
<mml:mo>−</mml:mo>
<mml:mi>L</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">]</mml:mo>
<mml:mo>,</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mi>r</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>,</mml:mo>
<mml:mo>...</mml:mo>
<mml:mo>,</mml:mo>
<mml:mfrac><mml:mrow><mml:mn>2</mml:mn>
<mml:mi>N</mml:mi>
</mml:mrow>
<mml:mi>L</mml:mi>
</mml:mfrac>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:math>
</disp-formula>
</p>
<p>Where <inline-formula><mml:math id="M12" name="1475-925X-9-52-i12" overflow="scroll"><mml:mrow><mml:msub><mml:mover accent="true"><mml:mi>W</mml:mi>
<mml:mo>¯</mml:mo>
</mml:mover>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</inline-formula>
is an array containing the moving average values of the importance values in column <italic>j</italic>
, then the updated <italic>importance </italic>
matrix is:</p>
<p><disp-formula id="bmcM12"><label>(12)</label>
<mml:math id="M13" name="1475-925X-9-52-i13" overflow="scroll"><mml:mrow><mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mo>:</mml:mo>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mi>m</mml:mi>
<mml:mi>a</mml:mi>
<mml:msubsup><mml:mi>x</mml:mi>
<mml:mrow><mml:mi>r</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mrow><mml:mn>2</mml:mn>
<mml:mi>N</mml:mi>
<mml:mo>/</mml:mo>
<mml:mi>L</mml:mi>
</mml:mrow>
</mml:msubsup>
<mml:mo>{</mml:mo>
<mml:msub><mml:mover accent="true"><mml:mi>W</mml:mi>
<mml:mo>¯</mml:mo>
</mml:mover>
<mml:mi>r</mml:mi>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>}</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>In the case of areas of very low importance beside very high importance ones, such as moving objects, then the final retargeted image will have some discontinuity artefacts. To remove any discontinuity in the <italic>importance </italic>
matrix, we convolve it with a Gaussian smoothing function [<xref ref-type="bibr" rid="B24">24</xref>
] which blurs the edges of the significant boundaries in the <italic>importance </italic>
matrix. The final updated matrix is:</p>
<p><disp-formula id="bmcM13"><label>(13)</label>
<mml:math id="M14" name="1475-925X-9-52-i14" overflow="scroll"><mml:mrow><mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mn>1</mml:mn>
<mml:mrow><mml:mn>2</mml:mn>
<mml:mi>π</mml:mi>
<mml:msup><mml:mi>σ</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
</mml:mrow>
</mml:mfrac>
<mml:msup><mml:mi>e</mml:mi>
<mml:mrow><mml:mo>−</mml:mo>
<mml:mo stretchy="false">(</mml:mo>
<mml:msup><mml:mi>i</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
<mml:mo>+</mml:mo>
<mml:msup><mml:mi>j</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>/</mml:mo>
<mml:mn>2</mml:mn>
<mml:msup><mml:mi>σ</mml:mi>
<mml:mn>2</mml:mn>
</mml:msup>
</mml:mrow>
</mml:msup>
<mml:mo>*</mml:mo>
<mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo>,</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>Figure <xref ref-type="fig" rid="F2">2(b), (c)</xref>
and <xref ref-type="fig" rid="F2">2(d)</xref>
, show the spatial, spatio-temporal and our modified <italic>importance </italic>
matrix, respectively.</p>
<sec><title>C) The shrinkability matrix</title>
<p>The <italic>importance </italic>
matrix defines which pixels in the source image are important and should be preserved in the retargeted image. In contrast, the <italic>shrinkability </italic>
matrix defines the relative extent to which pixels in the source image should be shrunk to achieve the target image. Because the values of all the pixels in each column of the <italic>importance </italic>
matrix are the same, then we compute the <italic>shrinkability </italic>
values of one row only and then replicate for the remaining rows. This considerably reduces the processing time. The <italic>shrinkability </italic>
of a pixel means the reduction in that pixel's width when reducing the source image width by K pixels. If we consider K equal to 1, which means reducing the size of the source image by only one pixel. Then the shrinking value of each pixel <italic>S</italic>
(<italic>j</italic>
)can be calculated from this equation:</p>
<p><disp-formula id="bmcM14"><label>(14)</label>
<mml:math id="M15" name="1475-925X-9-52-i15" overflow="scroll"><mml:mrow><mml:mi>S</mml:mi>
<mml:mtext>(</mml:mtext>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mfrac><mml:mn>1</mml:mn>
<mml:mrow><mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>*</mml:mo>
<mml:mstyle displaystyle="true"><mml:msubsup><mml:mo>∑</mml:mo>
<mml:mrow><mml:mi>j</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>M</mml:mi>
</mml:msubsup>
<mml:mn>1</mml:mn>
</mml:mstyle>
<mml:mo>/</mml:mo>
<mml:mi>W</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>j</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>The summation of <italic>S</italic>
(<italic>j</italic>
) over <italic>j </italic>
columns equals 1 if K is 1. For higher values of K, i.e.number or columns to be removed/shrunk, the summation of <italic>S</italic>
(<italic>j</italic>
) is equal to K. This equation is repeated for the rows.</p>
</sec>
</sec>
<sec><title>D) Scaling the Shrinkability matrix</title>
<p>To shrink the width of the source image by K columns, we can simply multiply the <italic>shrinkability </italic>
matrix by K. However, this will result in some of the pixels in the scaled matrix exceeding 1. This means that these pixels would be moved to the position of the left neighbour pixels. Such pixel movements can cause, unnatural artefacts in the retargeted image, and is known as the edge flipping problem [<xref ref-type="bibr" rid="B18">18</xref>
] which is described in Figure <xref ref-type="fig" rid="F4">4</xref>
.</p>
<fig id="F4" position="float"><label>Figure 4</label>
<caption><p><bold>Illustration of the edge flipping problem</bold>
. When the shrinkability value exceeds 1, the edge of the shrinkage pixel will move leftward past its left-hand edge. The number above each arrow indicates the value by which the pixel should be compressed.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-4"></graphic>
</fig>
<p>To solve this problem, we take the following steps:</p>
<p><disp-formula id="bmcM15"><label>(15)</label>
<mml:math id="M16" name="1475-925X-9-52-i16" overflow="scroll"><mml:mrow><mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mi>K</mml:mi>
<mml:mo>*</mml:mo>
<mml:mi>S</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p><disp-formula id="bmcM16"><label>(16)</label>
<mml:math id="M17" name="1475-925X-9-52-i17" overflow="scroll"><mml:mrow><mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mi>min</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>,</mml:mo>
<mml:mn>0.9</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p><disp-formula id="bmcM17"><label>(17)</label>
<mml:math id="M18" name="1475-925X-9-52-i18" overflow="scroll"><mml:mrow><mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>=</mml:mo>
<mml:mrow><mml:mo>(</mml:mo>
<mml:mrow><mml:mi>K</mml:mi>
<mml:mo>/</mml:mo>
<mml:mstyle displaystyle="true"><mml:mo>∑</mml:mo>
<mml:mrow><mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:mstyle>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
<mml:mo>*</mml:mo>
<mml:mo stretchy="false">(</mml:mo>
<mml:msup><mml:mi>S</mml:mi>
<mml:mo>'</mml:mo>
</mml:msup>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>J</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>We repeat equations (16) and (17) until the summation of S'(J) equals to K. To avoid the total removal of any pixels, the threshold value is set to 0.9 and not 1. This method is much faster and more accurate than the one described by Zhang et al. [<xref ref-type="bibr" rid="B18">18</xref>
], which applied a binary search method to find the best K<sub>o </sub>
value that is close to the required K.</p>
<p>Figure <xref ref-type="fig" rid="F2">2(e)</xref>
and <xref ref-type="fig" rid="F2">2(f)</xref>
, show the scaled shrinkage map when using the unmodified and modified <italic>importance </italic>
matrix, respectively.</p>
</sec>
<sec><title>E) Remapping algorithm</title>
<p>The scaled cumulative <italic>shrinkability </italic>
is used to resample the source image to the retarget image by using an algorithm suggested by Karl M. Fant [<xref ref-type="bibr" rid="B25">25</xref>
]. The algorithm is a 1 D method used in separable transformations defined in terms of forward mapping functions. It maps a limited line of discrete input pixel intensity values into a limited line of discrete output pixel intensity value. A full description can be found in Fant's paper, but we will summarise it briefly here.</p>
<p>The mapping is determined by a variable sizing factor of the output line in relation to the input line and by a position factor in relation to the output line. It can be characterized by the following formulation;</p>
<p><disp-formula><mml:math id="M19" name="1475-925X-9-52-i19" overflow="scroll"><mml:mrow><mml:mtext>Each consecutive output pixel</mml:mtext>
<mml:mo>=</mml:mo>
<mml:mtext>S</mml:mtext>
<mml:mo stretchy="false">(</mml:mo>
<mml:mtext>consecutive input pixels</mml:mtext>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>.</mml:mo>
</mml:mrow>
</mml:math>
</disp-formula>
</p>
<p>Where S is a variable scaling factor, a size factor of the output data in relation to the input data which changes from pixel to pixel in the same line and 1/S is the inverse of the size factor which indicates how much of an input pixel contributes to each successive output pixel.</p>
<p>Figure <xref ref-type="fig" rid="F2">2(g), (h)</xref>
and <xref ref-type="fig" rid="F2">2(i)</xref>
, show the retargeted image when using the ordinary interpolation method and when using the unmodified and modified <italic>importance </italic>
matrix, respectively.</p>
</sec>
<sec><title>F) Dynamic scene retargeting</title>
<p>To extend our algorithm into dynamic scene retargeting, we take into consideration some constraints from the previous frame. As previously mentioned, calculating the <italic>importance </italic>
map for each frame individually generates jittering artefacts in the retargeted video sequence. We therefore start by considering spatio-temporal <italic>importance </italic>
maps rather than considering the spatial importance alone (equation 4). If we relied purely on the spatio-temporal <italic>importance </italic>
map, seams could cross to the other side of a significant feature, causing significant jitter. To counteract this, movement of seams from one scene to the next need to be constrained.</p>
<p>We therefore calculate the seams of the first frame or first couple of frames, and then the actual locations of these seems are stored into an arbitrary matrix <italic>T </italic>
to be used in calculating the seams for the next frame. These seams' locations are adapted, for the forthcoming frames, if there are dynamics in the scene. If the objects in the frame are static then the locations of the seams will be the same, but if these objects move then the seam locations within the same areas through which the objects move around will also move to avoid crossing the moving objects.</p>
<p>For each seam, we first measure the difference between the energy of the stored seam path of the <italic>importance </italic>
matrix in the previous frame and the energy of the same path of the <italic>importance </italic>
matrix in the current frame. If the difference exceeds a threshold value <italic>Z</italic>
, we search for another seam in the whole frame that has lower energy, and if the difference does not exceed <italic>Z </italic>
then we search for a seam that might have energy less than the energy of the previous seam within a window of size Nx3 in the cumulative energy map <italic>M</italic>
(<italic>i</italic>
, <italic>j</italic>
). If the energy of the new seam is greater than the previous one then we keep the previous seam intact. A summary of this process is shown below:</p>
<p><disp-formula><mml:math id="M20" name="1475-925X-9-52-i20" overflow="scroll"><mml:mtable columnalign="left"><mml:mtr><mml:mtd><mml:mi>F</mml:mi>
<mml:mi>o</mml:mi>
<mml:mi>r</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>c</mml:mi>
<mml:mi>h</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>m</mml:mi>
<mml:mi>e</mml:mi>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>I</mml:mi>
<mml:mi>f</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mo>{</mml:mo>
<mml:mi>Σ</mml:mi>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>c</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mi>t</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mi>Σ</mml:mi>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>p</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>v</mml:mi>
<mml:mi>i</mml:mi>
<mml:mi>o</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>s</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>}</mml:mo>
<mml:mo>></mml:mo>
<mml:mi>Z</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>f</mml:mi>
<mml:mi>o</mml:mi>
<mml:mi>r</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>i</mml:mi>
<mml:mo>=</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>:</mml:mo>
<mml:mi>K</mml:mi>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>T</mml:mi>
<mml:mi>h</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>c</mml:mi>
<mml:mi>h</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>f</mml:mi>
<mml:mi>o</mml:mi>
<mml:mi>r</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>n</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>w</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>m</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>i</mml:mi>
<mml:mi>n</mml:mi>
<mml:mtext> </mml:mtext>
<mml:msub><mml:mi>M</mml:mi>
<mml:mrow><mml:mi>c</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mi>t</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>E</mml:mi>
<mml:mi>l</mml:mi>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>S</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>c</mml:mi>
<mml:mi>h</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>f</mml:mi>
<mml:mi>o</mml:mi>
<mml:mi>r</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>n</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>w</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>m</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>i</mml:mi>
<mml:mi>n</mml:mi>
<mml:mtext> </mml:mtext>
<mml:msub><mml:mi>M</mml:mi>
<mml:mrow><mml:mi>c</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mi>t</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>−</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo>,</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>,</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>+</mml:mo>
<mml:mn>1</mml:mn>
<mml:mo stretchy="false">)</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>I</mml:mi>
<mml:mi>f</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>Σ</mml:mi>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>c</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mi>t</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:msub><mml:mi>T</mml:mi>
<mml:mrow><mml:mi>n</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>w</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo>></mml:mo>
<mml:mi>Σ</mml:mi>
<mml:msub><mml:mi>W</mml:mi>
<mml:mrow><mml:mi>c</mml:mi>
<mml:mi>u</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mi>t</mml:mi>
</mml:mrow>
</mml:msub>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
<mml:mo stretchy="false">)</mml:mo>
</mml:mtd>
</mml:mtr>
<mml:mtr><mml:mtd><mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mtext> </mml:mtext>
<mml:mi>T</mml:mi>
<mml:mi>h</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>n</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>u</mml:mi>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>o</mml:mi>
<mml:mi>l</mml:mi>
<mml:mi>d</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>m</mml:mi>
<mml:mtext> </mml:mtext>
<mml:mi>T</mml:mi>
<mml:mo stretchy="false">(</mml:mo>
<mml:mi>i</mml:mi>
<mml:mo stretchy="false">)</mml:mo>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:math>
</disp-formula>
</p>
<p>By iteration, we have found that the best threshold value Z is 20% greater than the energy sum of the previous seam. Lower values of Z result in more frequent re-evaluations of the seams. This is more sensitive but results in jitter effects. Higher values of Z, increases stability with respect to jitter, but could result in distortion of the geometry of significant features.</p>
</sec>
<sec><title>G) Downsampling to improve computational efficiency</title>
<p>There is one additional advantage of our algorithm over the <italic>seam carving </italic>
method. In the <italic>seam carving </italic>
approach, the process of generating the energy map and searching for the seams should be applied on the full size of the image to remove the seams repetitively. However, in our approach we generate seams to create an <italic>importance </italic>
map rather than removing them from the scene. Hence, we can resize the source image into smaller one and generate the <italic>importance </italic>
matrix from this downsampled image. After that, we resize the <italic>importance </italic>
matrix into the original size of the source image. For example, if we down sized the source image by half, the overall processing time will be reduced by more than 60%, as most of the processing time is spent in generating and updating the <italic>importance </italic>
matrix. This approach works well for high contrast scenes, though less well for more complex lower contrast scenes.</p>
</sec>
</sec>
<sec><title>Evaluation of Performance</title>
<p>The evaluation process is divided into two sections; synthetic evaluation based on testing the performance of our algorithm compared to other retargeting methods, and evaluations based on testing our algorithm on real subjects.</p>
<p>From a comprehensive review of the literature it is clear that there is a lack of objective assessment tools to test efficacy of image rescaling algorithms. We have therefore generated a synthesized video (600 × 256) containing a three moving text boxes of the sentence "Peripheral Vision Test". The three boxes moved throughought the video space, from left to right and top to bottom. Gaussian Noise was added to investigate the effect of image texture complexity. Figure <xref ref-type="fig" rid="F5">5</xref>
, shows a snapshot of the synthesized video.</p>
<fig id="F5" position="float"><label>Figure 5</label>
<caption><p><bold>Snapshot of our synthetic test video</bold>
. Showing three text boxes and blue alignment marker in the four directions; top, down, right and left.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-5"></graphic>
</fig>
<p>If not done so already with automatic gain control, the video's frames luminance intensities are normalized between 0 and 255. We then applied our video retargeting algorithm for different retargeting sizes on this video file, from 20% to 62% of the original size, and did the same for the <italic>seam carving </italic>
and <italic>shrinkability </italic>
methods. The efficiency of the three algorithms was compared with respect to the ordinary video resizing by interpolation. For comparison, and to avoid subjective human interpretation, we measured three objective parameters:</p>
<sec><title>1) Recognisability with respect to compression ratio</title>
<p>To measure this parameter we used optical character recognition (OCR) to count the number of correctly recognized words along the whole video frames. We developed our OCR interpreter based on the character template matching approach [<xref ref-type="bibr" rid="B26">26</xref>
].</p>
</sec>
<sec><title>2) Compressibility</title>
<p>We measured the ratio between the average sizes of the three text boxes; when using the three algorithms, and when using the interpolation method.</p>
</sec>
<sec><title>3) Alignment</title>
<p>In the original video file we inserted four small blue circles (top, bottom, left and right). We measured the average vertical and horizontal displacement of these circles along the whole video frames to determine alignment changes after processing.</p>
<p>In addition, our algorithm was objectively and subjectively evaluated on real subjects. In the objective evaluation, our algorithm was tested on 20 volunteers using a simulated tunnel vision. Two pair of goggles, covering the whole eye area, was artificially painted, apart from an aperture, to simulate 10° and 20° tunnel vision effects. Objective testing was divided into 4 sections.</p>
<p>The first section was to assess the affect of our algorithm on speed of recognition. 26 images were displayed to the participants (13 with original size and 13 retargeted to 53% using our method). Images were projected to the participants using the NEC LT280 projector with resolution of 1024 × 768 and maximum projection brightness (At a distance of 2 m from the projection wall) of 2500 Lumens, in a darkened room. The projected area for the original image was (225 × 180 cm) which equivalent to 70° field of view. The 13 images included 3 synthetic images with colored shapes of squares, triangles and circles. The remaining 10 real-world images included football players with colored shirts on a grass background. Participants were asked to count the number of certain colored shapes (synthetic images) or shirts (real-world images) and allowed to move their head freely. The time taken to count these objects was measured for both of the original and retargeted images. This experiment was done for two simulated tunnel vision degrees; 11° and 22° field of view (FOV), respectively. Of the 20 participants, 6 were used as control subjects to do the same task without wearing the goggles. Those 6 subjects did both the whole test and the control part. Samples of the projected images are presented in the Appendix.</p>
<p>The second section of the test measured the efficiency of our method compared to the ordinary (linear) rescaling method. Three different images were compressed to 10%, 20% 40% and 50% of the original size using standard rescaling and our seam assisted shrinkability algorithm. The images were projected to the participants starting from the smallest size (10%) to the largest, and the ordinary rescaled images were displayed before our retargeted images. Participants were able to move their head freely and asked to count the number of colored shapes and the number of people in the images. The percentages of recognized objects were measured with respect to the total number of objects (total of 21 objects in the 3 images) in each scale for the ordinary resized images and our retargeted images.</p>
<p>The third section determined to what extent the compression algorithm could increase the effective field of view. The participants were asked to watch 3 video files (1 synthetic and 2 real-world) which included specific actions and objects. The participants were asked to count the number of times they perceived certain events in their peripheral field. The test was repeated for 40% retargeted versions using our algorithm. The percentage of detected objects was measured with respect to the total number of objects and actions (a total of 17 objects and actions occurred in the 3 video files) for both, the original and retargeted video. In two of the videos, the participants were asked to follow the moving object and in the third one they were free to move their heads. A Full description for the three video files used in this section is explained in the Appendix.</p>
<p>The last section measured the efficiency of our method over the ordinary rescaling method in recognizing certain actions in video. We compressed video files to 25%, 35% and 45%, respectively, of the original size using the ordinary rescaling and our method. The video was projected to the participants starting from the smallest size (30%) to the larger one, and the ordinary rescaled one is displayed before our retargeted one. The subjects were allowed to move heads freely. The percentages of recognized actions were measured with respect to the total number of actions (total of 4 actions) in each scale for the ordinary resized and our retargeted version. The description of the 4 actions is presented in more detail in the Appendix.</p>
<p>The subjective test of this part is divided into two sections. Firstly, the performance of our seam assisted shrinkability method was subjectively compared to the seam carving and shrinkability only methods on healthy subjects. We then separately performed a comparison of our seam assisted shrinkability algorithm with ordinary (linear) resizing. This second study is to explore the tradeoffs' between improved object sizes with inevitable levels of jitter. 6 video files were used in this evaluation, which were each retargeted with the three algorithms plus retargeting them with the ordinary resizing technique. The first video sequence consisted of four subjects passing a tennis ball to each other while they are standing and not moving. In the second video sequence, 5 subjects pass a basket ball while they are moving in a circular form. The third video consists of a subject moving (in the middle of the scene) towards the screen from a distance while three other subjects enter and exit from the frame. In the fourth video, a subject is playing with golf balls, so the motion in this video was considerably slow. The fifth video consists of a subject skating on the water. The final video consisted of eight subjects dancing in a small room. This was the busiest of the 6 videos.</p>
<p>Table <xref ref-type="table" rid="T1">1</xref>
shows the dimensions of each video file and the rate of compression applied. For the first part of the test, the retargeted videos were shown to 14 normal subjects who were asked to select the most subjectively preferable. For the second part, subjects were asked to select the most preferable one between the most preferable from the first part and the ordinary resized videos. Subjects were asked to consider their judgement based on the degree of object distortion, and jitter between frames.</p>
<table-wrap id="T1" position="float"><label>Table 1</label>
<caption><p>Sizes of the original and retargeted videos used in the preference experiment</p>
</caption>
<table frame="hsides" rules="groups"><thead><tr><th align="center">Video number</th>
<th align="center">Original size</th>
<th align="center">Retargeted size</th>
<th align="center">% Compression</th>
</tr>
</thead>
<tbody><tr><td align="center">1</td>
<td align="center">240 × 320</td>
<td align="center">240 × 180</td>
<td align="center">43.75</td>
</tr>
<tr><td align="center">2</td>
<td align="center">240 × 320</td>
<td align="center">240 × 200</td>
<td align="center">37.5</td>
</tr>
<tr><td align="center">3</td>
<td align="center">240 × 320</td>
<td align="center">240 × 200</td>
<td align="center">37.5</td>
</tr>
<tr><td align="center">4</td>
<td align="center">400 × 700</td>
<td align="center">400 × 300</td>
<td align="center">57.14</td>
</tr>
<tr><td align="center">5</td>
<td align="center">210 × 320</td>
<td align="center">210 × 160</td>
<td align="center">50</td>
</tr>
<tr><td align="center">6</td>
<td align="center">200 × 420</td>
<td align="center">200 × 200</td>
<td align="center">52.38</td>
</tr>
</tbody>
</table>
<table-wrap-foot><p>All the P values mentioned in the next sections are calculated based on the one tail t-Test.</p>
</table-wrap-foot>
</table-wrap>
</sec>
</sec>
<sec><title>Results</title>
<p>We present the results in two sections; the synthetic performance of our algorithm compared existing methods, and results obtained from human testing with simulated tunnel vision.</p>
<sec><title>A) Results from synthetic testing</title>
<p>Figure <xref ref-type="fig" rid="F6">6</xref>
, shows the performance of our algorithm compared to the <italic>seam carving </italic>
and the <italic>shrinkability </italic>
methods in respect to the three measuring parameters mentioned previously. The left column shows the performance for low added noise (σ = 0.025), and the right for high added noise (σ = 0.45), respectively. We can see that our algorithm has the highest OCR recognition values over the range of compression rates. This is because it combines the advantages of preserving the size of the text boxes, which are the foreground objects in this case. Additionally, it maintains the connectivity of these objects without distortion. From Figure <xref ref-type="fig" rid="F6">6</xref>
middle row, we can see that <italic>seam carving </italic>
has the highest compressibility, but this efficiency in compression is to the cost of increased distortion, and reduced connectivity and stability of the objects as shown in Figure <xref ref-type="fig" rid="F7">7</xref>
. This is because seams tend to go through higher energy pixels, if higher compression is required. Figure <xref ref-type="fig" rid="F6">6</xref>
last row, compares the number of unaligned pixels for the three methods. The <italic>seam carving </italic>
alone causes more zig-zag and misalignment effects than our algorithm and the <italic>shrinkability </italic>
method while increasing the compression rate.</p>
<fig id="F6" position="float"><label>Figure 6</label>
<caption><p><bold>A comparison between the performance of seam carving, shrinkability, and our method by measuring the three parameters</bold>
: The OCR (top row), the compressibility (middle row) relative to the ordinary interpolation, and the Zig-Zag effect. The left column shows the performance when σ of the added Gaussian noise (simulation to the contrast between the foreground and background) is 0.025, and the right column when σ is 0.45.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-6"></graphic>
</fig>
<fig id="F7" position="float"><label>Figure 7</label>
<caption><p><bold>Comparison between the retargeting of the frame by 50% of its original size when using</bold>
. The shrinkability algorithm (a), the seam carving algorithm (b), and our seam shrinking algorithm (c).</p>
</caption>
<graphic xlink:href="1475-925X-9-52-7"></graphic>
</fig>
<p>Figure <xref ref-type="fig" rid="F6">6</xref>
and Figure <xref ref-type="fig" rid="F8">8</xref>
, show the effect of increasing the σ of the added Gaussian noise on the efficiency of our method compared to the <italic>seam carving </italic>
and the <italic>shrinkability </italic>
methods in respect to the OCR, compressibility and unalignment measurements, respectively. From the top row of Figure <xref ref-type="fig" rid="F6">6</xref>
and Figure <xref ref-type="fig" rid="F8">8</xref>
, we see that the OCR efficiency drops sharply with increasing the noise levels in the case of <italic>seam carving </italic>
method. This is because adding more noise blurs the foreground/background contrast and hence the energy map will tend to be smoothed with slightly equal weights. In the case of the <italic>shrinkability </italic>
method, the drop in the OCR efficiency starts to increase sharply when a compression rate of more than 45% is used. Our method shows a slight drop in the OCR efficiency with compression, but not as much as the other two methods.</p>
<fig id="F8" position="float"><label>Figure 8</label>
<caption><p><bold>The effect of increasing σ of the added Gaussian noise (to simulate the variation on contrast between the foreground and background)</bold>
. When 25% compression rate from the original size is required (left column) and 62% compression rate (right). The first row is for the OCR measurement, the second row is for the compressibility measurement, and the last row is for the Zig-Zag or unalignment measurement.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-8"></graphic>
</fig>
<p>The middle row of Figure <xref ref-type="fig" rid="F8">8</xref>
shows the effect of increasing the noise level on the compressibility performance. When a small compression rate is required and the noise level is low (which means the background gradient is low), our seam assisted shrinkability method efficiency is high. This is because the seams will be accumulated in the low gradient areas, causing these areas to be shrunk more. Increasing the noise level (gradient of background becomes high) drops the compressibility performance in all algorithms. Our algorithm performs better than shrinkability only, but starts to approach this latter algorithm at high noise levels during lower compression.</p>
<p>Also from the last row of Figure <xref ref-type="fig" rid="F6">6</xref>
and Figure <xref ref-type="fig" rid="F8">8</xref>
, we see that the number of unaligned pixels tends to decrease with increasing the noise level and when high compression rates are required. This is because increasing the noise level gives the pixels increasingly equal energy, resulting in some of the seams crossing into the text box. Hence, the seams will tend to go straight from down to top causing less misalignment effect.</p>
<p>Figure <xref ref-type="fig" rid="F9">9</xref>
, shows the retargeting of different images when using the traditional image resize, <italic>seam carving</italic>
, <italic>shrinkability </italic>
and our seam assisted shrinkability method. Retargetting was performed on the horizontal direction only for these images to compare effects. <italic>Seam carving </italic>
results in effective scene compression up to a certain target width. However, this is on the cost of removing parts of the foreground objects, causing severe distortions in these objects. Since seams have irregular shapes, the distortion is unpredictable. Alternatively, our proposed algorithm compensates these distortions by compressing the pixels instead of removing them.</p>
<fig id="F9" position="float"><label>Figure 9</label>
<caption><p><bold>A comparison between different image retargeting techniques and our method</bold>
. From left to right: original, interpolation, shrinkability, seam carving, and our seam assisted shrinkability method. All the images are retargeted to 62% of the original size.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-9"></graphic>
</fig>
</sec>
<sec><title>B) Results from human trials</title>
<p>Results from the real subjects are divided into objective and subjective results. Objective's results are divided into 4 sections, as described in the evaluation section. The first two sections are for the still images and the third and fourth sections are for the dynamic scenes. The time taken for counting the number of objects within the 26 images is shown in Figure <xref ref-type="fig" rid="F10">10</xref>
. We can see that there is a significant reduction (P < 0.001) in the time taken to count the objects of the retargeted images by approximately 50% in the case of 11° FOV goggle. Also there is a significant reduction (P < 0.001) in the time by 35% when using the 22° FOV. Even then, there was a relatively significant (P < 0.035) difference between the times taken in the original sizes and the retargeted ones in the control subjects, which due to the peripheral vision effect. As the participants were still needs to move their eyes around the images while counting, while in the retargeted images they do not need to move their eyes.</p>
<fig id="F10" position="float"><label>Figure 10</label>
<caption><p><bold>The efficiency of the seam assisted shrinkability method on search time</bold>
. We simulated the effect of tunnel vision by asking users to wear blacked out goggles with apertures corresponding to 11° and 22° FOV. Greater improvement on search time improves as the tunnel vision worsens. Mid-Late stage RP typically expresses tunnel vision in the 5° to 20° range. The error bars represent the standard error of the data.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-10"></graphic>
</fig>
<p>Figure <xref ref-type="fig" rid="F11">11</xref>
, shows the efficiency of our compression method over the ordinary (linear) rescaling method, when compressing the images to 10%, 20%, 40% and 50% of their original size. We can see that, using our compression technique, more than 72% of the objects in the images are still recognizable when compressing the image into 10% of the original size, compared to only 28% when linearly compressed. This gives more than 150% enhancement in recognition over the ordinary resizing. The efficiency gap became very small at 50% compression.</p>
<fig id="F11" position="float"><label>Figure 11</label>
<caption><p><bold>Recognition variation with spatial compression</bold>
. The efficiency of retargeting the images using our seam assisted shrinkability method compared to the ordinary resizing method in recognizing objects, when scenes were compressed into 10%, 20%, 40% and 50%, respectively, of their original sizes. The error bars represent the standard error of the data.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-11"></graphic>
</fig>
<p>In the case of dynamic motion, Figure <xref ref-type="fig" rid="F12">12</xref>
shows the efficiency of detecting and counting moving objects when the scene retargeted to the field of view. As shown from the figure, there is a significant difference (P < 0.001) between the counted objects in the original videos and the retargeted videos. This gives an enhancement of object detection by more than 136% when retargeting the scene.</p>
<fig id="F12" position="float"><label>Figure 12</label>
<caption><p><bold>The efficiency of detecting objects</bold>
. The efficiency of detecting objects when retargeting dynamic scene into 40% of the original size using our seam assisted shrinkability method compared to the original size. The error bars represent the standard error of the data.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-12"></graphic>
</fig>
<p>Similar to the observations from Figure <xref ref-type="fig" rid="F11">11</xref>
, Figure <xref ref-type="fig" rid="F13">13</xref>
shows the efficiency of our compression method over the linear rescaling method, when compressing the dynamic scene into 25%, 35% and 45%, respectively, of their original size. More than 77% of the actions are still recognizable at 25% of the original video size when retargeted using our method, compared to only 37% when linearly resized. This gives more than 100% enhancement in recognition process over the ordinary resizing. However, the efficiency gap tends to decrease when reducing the compression rate.</p>
<fig id="F13" position="float"><label>Figure 13</label>
<caption><p><bold>The efficiency of recognizing actions</bold>
. The efficiency of recognizing actions when compressing the dynamic scene into 25%, 35, and 45% of the original size, using the ordinary resizing method and our method. The error bars represent the standard error of the data.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-13"></graphic>
</fig>
<p>Results obtained from the subjective preference test of our method, the <italic>seam carving </italic>
and <italic>shrinkability </italic>
only methods are shown in Figure <xref ref-type="fig" rid="F14">14</xref>
. The results show the average preference over the 14 subjects for the 6 dynamic movies. Overall, the retargeted videos by the seam assisted shrinkability method were preferred by 93% compared to the shrink and seam only methods, 7% and 0% respectively. This result shows good evidence on the efficiency of our proposed method over the previous two methods. In the second part of the subjective test, our seam assisted shrinkability method was compared with the ordinary resized videos. Figure <xref ref-type="fig" rid="F15">15</xref>
shows the result of this evaluation. Overall, our method was preferred by 75% compared to 22.22 of the ordinary resized videos. For the remaining 2.78%, the subjects couldn't see much difference between the two retargeting methods. These results show that the low levels of jitter resulting from our retargeting method is acceptable given the added benefit.</p>
<fig id="F14" position="float"><label>Figure 14</label>
<caption><p><bold>Subjective preference test</bold>
. The preference test is between our method and the seam and shrikability methods. The test was done on 14 subjects showed to them 6 retargeted videos using the three methos.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-14"></graphic>
</fig>
<fig id="F15" position="float"><label>Figure 15</label>
<caption><p><bold>Subjective preference test</bold>
. The preference test is between our method and the ordinary resized method.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-15"></graphic>
</fig>
</sec>
</sec>
<sec><title>Discussion</title>
<p>The synthetic results indicate that our method is superior to both the standard resizing method and previous seam carving and shrinkibility retargeting methods. Simulated patient trials indicate that greatest benefit is achieved in those of remaining field of view less than 20°. The algorithm would therefore be also useful in retinal prosthetic devices [<xref ref-type="bibr" rid="B27">27</xref>
] which will in initial implementations only return a limited visual field. The recent optogenetic approaches [<xref ref-type="bibr" rid="B27">27</xref>
] look particularly promising and could potentially return higher resolutions. However, assuming the technology can approach a level of development of present head mounted displays for virtual reality, the fields of few will still be limited to typically 40°. Thus some form of compression would greatly assist such users. As the visual acuity may still be low, scene simplification and enhancement operations may additionally be needed [<xref ref-type="bibr" rid="B28">28</xref>
].</p>
<p>Importantly for application to augmented reality visual aid devices, this algorithm can dynamically compress video sequences without knowledge of future frames. While low levels of jitter are ever present, we believe that at moderate compression rates the jitter is sufficiently low to be acceptable to our target patient group. Although this has not yet been tested in patients, subjective tests in healthy subjects show indicate efficacy. The level of jitter increases when large compression is required in highly dynamic and busy environments, such as the one shown in Figure <xref ref-type="fig" rid="F16">16</xref>
. This figure shows a snapshot from a video of guys playing with a basketball in a very small area. For such environment, a 25% compression from the original size can give acceptable results. However, in less crowded environments, up to 60% compression can be achieved with low levels of jitter. We would thus envisage patient's being able to control levels of compression rate according to the surrounding environment and personal preference.</p>
<fig id="F16" position="float"><label>Figure 16</label>
<caption><p><bold>Limitation of the algorithm for busy environment</bold>
. (a) Snapshot of a busy video for guys playing with basketball in a very small area. (b) Shows a 25% compression on it, when compressed linearly (right) and with our retargeting algorithm (left).</p>
</caption>
<graphic xlink:href="1475-925X-9-52-16"></graphic>
</fig>
<p>Ultimately, this work has shown efficacy in scenes whereby the camera is static and objects in the scene move. In this situation creating an energy map for the motion energy is straight forward and obtained from the movement of objects across the frames. However, in the situation where the camera moves relative to the environment, the movements of all features would need to be subtracted from the background movement. This background movement would then have to be calculated from optic flow analysis on the images or from accelerometer motion sensors.</p>
<p>A further point to be considered is that this presented algorithm has been designed for monocular vision. In binocular cases, the situation is more complex as slight differences in left and right scenes may vary the compression such that objects to not appear to overlap in 3D space. We will thus look to further develop this technique in future for binocular use.</p>
<p>Key to implementability of any retargeting algorithm is its function on portable processing systems at full video frame rate. We used the Matlab platform and the C language to build our seam assisted shrinkability algorithm. The processing was implemented on a desktop computer; HP workstation XW4600 with a 2.6 GHz Intel Core Quad processor. We did not use any GPU assisted processes and thus all the operations are processed through the CPU/FPU. Table <xref ref-type="table" rid="T2">2</xref>
shows the processing time of the major parts in the algorithm when retargeting one frame of size 352 × 288 by 43%. The table shows the processing time when downsampling the input frames into different scales to calculate the <italic>importance </italic>
map.</p>
<table-wrap id="T2" position="float"><label>Table 2</label>
<caption><p>Downsampling effect on the processing time</p>
</caption>
<table frame="hsides" rules="groups"><thead><tr><th align="center">Downsampling by</th>
<th align="center" colspan="4">Processing time (sec)</th>
</tr>
<tr><th></th>
<th align="center">Importance map</th>
<th align="center">Shrinkability & Fant</th>
<th align="center">Other functions</th>
<th align="center">Total</th>
</tr>
</thead>
<tbody><tr><td align="center">1</td>
<td align="center">0.8</td>
<td align="center">0.02</td>
<td align="center">0.03</td>
<td align="center">0.85</td>
</tr>
<tr><td align="center">2</td>
<td align="center">0.17</td>
<td align="center">0.02</td>
<td align="center">0.03</td>
<td align="center">0.22</td>
</tr>
<tr><td align="center">4</td>
<td align="center">0.05</td>
<td align="center">0.02</td>
<td align="center">0.03</td>
<td align="center">0.1</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>We can see that the processing time of the <italic>importance </italic>
map drops approximately by 4 when downsampling the input frames by half of the original size. However, the time taken in generating the <italic>shrinkability </italic>
map and remapping by Fant's algorithm is fixed because these processes applied on the whole size of the input frame.</p>
<p>Currently, all of the algorithm's parts are running serially on the computer CPU. However, some parts of the algorithm can be optimized to run in parallel on a graphic processing unit (GPU) or even on a portable field programmable gated array (FPGA) device [<xref ref-type="bibr" rid="B29">29</xref>
]. For example, most of the <italic>importance </italic>
map parts are based on the convolution kernels which can be implemented in parallel. Not only can the importance map be parallelized, but also the process of generating and scaling the <italic>shrinkability </italic>
map. Such parallel processing can be performed in portable GPU or FPGA devices. Because the Fant's algorithm remaps each row individually on the retarget frame, it is also possible to implement this in parallel architecture. In future work we hope to implement this algorithm on a portable parallel processing platform so as to perform real time RP patient trials. As is generally accepted in the graphics processing community, parallel processing using GPU architectures can speed up the processing time in the range of 10-100× depending on the level of parallelism of the program. Although 25frames per second is sufficient for video rate, 50 frames per second is widely accepted as the minimum to reduce strain through motion blur. We believe this target can be achieved in portable GPU architectures.</p>
</sec>
<sec><title>Conclusions</title>
<p>In this article we have described a novel content aware scene retargeting technique developed for patients suffering from retinitis pigmentosa. Our approach tackles the shortcomings of the two current techniques for image retargeting (<italic>seam carving </italic>
and <italic>image shrinkability)</italic>
. In particular we have improved upon issues such as discontinuity artefacts and jitter in real time video sequences.</p>
<p>We combined the advantages of both techniques, by designing a novel <italic>importance </italic>
map which gives the pixels along the seams paths lower importance values. Then we used the modified <italic>importance </italic>
map to build a <italic>shrinkability </italic>
matrix to shrink the pixels according to their shrinkage values.</p>
<p>Results show the robustness of our approach compared to the <italic>seam carving </italic>
and image <italic>shrinkability </italic>
techniques in preserving the scene contents intact and in the compressibility performance.</p>
</sec>
<sec><title>Consent</title>
<p>Written consent was obtained from the participants for publication of this paper and accompanying images. A copy of the written consent is available for review by the Editor-in-Chief of this journal.</p>
</sec>
<sec><title>Competing interests</title>
<p>The authors declare that they have no competing interests.</p>
</sec>
<sec><title>Authors' contributions</title>
<p>WA developed the algorithms, TT developed the image shrinkability component. WA and PD performed the data analysis and manuscript writing. All authors read and approved the final manuscript</p>
</sec>
<sec><title>Appendix: Description of the trials</title>
<p>To evaluate the efficiency of our method in reducing the searching time for specific objects, we know that the selected images in the trials should have the same number of objects before and after retargeting. It is not easy to select very similar images that have the same complexity and number of objects to compare between them. For that we selected the images of the players in the playground to be used in this trial, which eliminate the effect of memory. Figure <xref ref-type="fig" rid="F17">17(a)</xref>
, shows a sample of the used images in the first section before and after retargeting.</p>
<fig id="F17" position="float"><label>Figure 17</label>
<caption><p><bold>Samples from the trial</bold>
. Samples of the images used in the first section of the test (a). And in the second section (b), where the image is compressed to 10% (right) and 20% (left) of the original size, respectively. The left image in each pair of images in (b) is the one compressed linearly and the right one is the retargeted using our method.</p>
</caption>
<graphic xlink:href="1475-925X-9-52-17"></graphic>
</fig>
<p>Figure <xref ref-type="fig" rid="F17">17(b)</xref>
, shows sample from the second part of the test. The lowest size of the image is displayed first to the participant to count the number of objects in the image (objects here is the total number of people), as shown in the right pair of images. Then the larger size is displayed to count the number of objects, as shown in the left pair. The image to the left in each pair is the one resized linearly and the one to the right is retargeted using our method, respectively.</p>
<p>In the third section of the test, the first video shown to the participant includes a blue arrow moving from the left side to the right hand side. Participants are asked to focus on this moving arrow and count the number of blue squares appearing in their FOV, which are 3 blue squares in total. The second video file includes 2 pairs of persons passing two small tennis balls to each other. The participants are asked to count the number of passes of the tennis balls, which are 11 passes. In this video, the participants were free to scan the whole video while counting. In the third file, the participants are asked to focus on a walking person and to count the number of people appearing in their FOV, which are 3 in total. A 40% of the original size retargeted versions of these videos were shown to the participants to count the same objects. Figure <xref ref-type="fig" rid="F18">18(a, b)</xref>
, shows a snapshot of the second video file (people playing with a tennis ball) before and after retargeting, respectively.</p>
<fig id="F18" position="float"><label>Figure 18</label>
<caption><p><bold>Snapshots from the dynamic scenes</bold>
. Snapshots from the second video file in the third section of the test, before (a) and after (b) retargeting, respectively. Snapshots from the video used in the forth section of the test, where the video is compressed into 25% of its original size using linear resizing (c) and our retargeting method (d).</p>
</caption>
<graphic xlink:href="1475-925X-9-52-18"></graphic>
</fig>
<p>The last section in the test includes only one video which compressed into 25%, 35, and 45% of its original size using linear rescaling and our retargeting method. The movie includes 4 persons moving around while doing specific actions (reading, drinking, eating a banana, and holding a child). The participants are asked to recognize the actions of these persons in each scale. Figure <xref ref-type="fig" rid="F18">18(c, d)</xref>
, shows a snapshot from the 25% version of the video, when linearly resized and when using our retargeting method, respectively.</p>
</sec>
<sec><title>Note</title>
<p>NB: All the images and videos included in this manuscript have been taken by the author.</p>
</sec>
<sec><title>Acknowledgements</title>
<p>We would like to acknowledge and thank The University of London Central Research Fund (AR/CRF/B), the Royal Society Research fund, the National Institute for Health research (BRC) fund and the EPSRC (F029241) for supporting this research. Also Mr. Walid Al-Atabany would like to thank the Egyptian government, who are sponsoring him for his PhD. We would also like to thank Dr. Muhammad Ali Memon and Dr. Susan Downes for our collaborations on past, present and future patient trials in this augmented vision area.</p>
</sec>
</body>
<back><ref-list><ref id="B1"><mixed-citation publication-type="journal"><name><surname>Foster</surname>
<given-names>A</given-names>
</name>
<name><surname>Resnikoff</surname>
<given-names>S</given-names>
</name>
<article-title>The impact of Vision 2020 on global blindness</article-title>
<source>Eye</source>
<year>2005</year>
<volume>19</volume>
<fpage>1133</fpage>
<lpage>1135</lpage>
<pub-id pub-id-type="doi">10.1038/sj.eye.6701973</pub-id>
<pub-id pub-id-type="pmid">16304595</pub-id>
</mixed-citation>
</ref>
<ref id="B2"><mixed-citation publication-type="journal"><name><surname>Hamel</surname>
<given-names>C</given-names>
</name>
<article-title>Retinitis pigmentosa</article-title>
<source>Orphanet Journal of Rare Diseases</source>
<year>2006</year>
<volume>1</volume>
<fpage>40</fpage>
<pub-id pub-id-type="doi">10.1186/1750-1172-1-40</pub-id>
<pub-id pub-id-type="pmid">17032466</pub-id>
</mixed-citation>
</ref>
<ref id="B3"><mixed-citation publication-type="journal"><name><surname>Vargas-Martín</surname>
<given-names>F</given-names>
</name>
<name><surname>Peláez-Coca</surname>
<given-names>MD</given-names>
</name>
<name><surname>Ros</surname>
<given-names>E</given-names>
</name>
<name><surname>Diaz</surname>
<given-names>J</given-names>
</name>
<name><surname>Mota</surname>
<given-names>S</given-names>
</name>
<article-title>A generic real-time video processing unit for low vision</article-title>
<source>International Congress Series</source>
<year>2005</year>
<volume>1282</volume>
<fpage>1075</fpage>
<lpage>1079</lpage>
<pub-id pub-id-type="doi">10.1016/j.ics.2005.05.107</pub-id>
</mixed-citation>
</ref>
<ref id="B4"><mixed-citation publication-type="journal"><name><surname>Fullerton</surname>
<given-names>M</given-names>
</name>
<name><surname>Peli</surname>
<given-names>E</given-names>
</name>
<article-title>Post Transmission Digital Video Enhancement for People with Visual Impairments</article-title>
<source>J Soc Inf Disp</source>
<year>2006</year>
<volume>14</volume>
<fpage>15</fpage>
<lpage>24</lpage>
<pub-id pub-id-type="doi">10.1889/1.2166829</pub-id>
<pub-id pub-id-type="pmid">16823464</pub-id>
</mixed-citation>
</ref>
<ref id="B5"><mixed-citation publication-type="journal"><name><surname>Wolffsohn</surname>
<given-names>JS</given-names>
</name>
<name><surname>Mukhopadhyay</surname>
<given-names>D</given-names>
</name>
<name><surname>Rubinstein</surname>
<given-names>M</given-names>
</name>
<article-title>Image enhancement of real-time television to benefit the visually impaired</article-title>
<source>Am J Ophthalmol</source>
<year>2007</year>
<volume>144</volume>
<fpage>436</fpage>
<lpage>440</lpage>
<pub-id pub-id-type="doi">10.1016/j.ajo.2007.05.031</pub-id>
<pub-id pub-id-type="pmid">17632067</pub-id>
</mixed-citation>
</ref>
<ref id="B6"><mixed-citation publication-type="other"><name><surname>Atabany</surname>
<given-names>W</given-names>
</name>
<name><surname>Degenaar</surname>
<given-names>P</given-names>
</name>
<article-title>A Robust Edge Enhancement Approach for Low Vision Patients Using Scene Simplification</article-title>
<source>Cairo International Biomedical Engineering Conference CIBEC</source>
<year>2008</year>
<fpage>1</fpage>
<lpage>4</lpage>
<comment>full_text</comment>
</mixed-citation>
</ref>
<ref id="B7"><mixed-citation publication-type="journal"><name><surname>Bennett</surname>
<given-names>J</given-names>
</name>
<article-title>Gene therapy for retinitis pigmentosa</article-title>
<source>Current Opinion in Molecular Therapeutics</source>
<year>2000</year>
<volume>2</volume>
<fpage>420</fpage>
<lpage>425</lpage>
<pub-id pub-id-type="pmid">11249772</pub-id>
</mixed-citation>
</ref>
<ref id="B8"><mixed-citation publication-type="journal"><name><surname>Peli</surname>
<given-names>E</given-names>
</name>
<name><surname>Luo</surname>
<given-names>G</given-names>
</name>
<name><surname>Bowers</surname>
<given-names>A</given-names>
</name>
<name><surname>Rensing</surname>
<given-names>N</given-names>
</name>
<article-title>22.4: Invited Paper: Augmented Vision Head-Mounted Systems for Vision Impairments</article-title>
<source>SID Symposium Digest of Technical Papers</source>
<year>2007</year>
<volume>38</volume>
<fpage>1074</fpage>
<lpage>1077</lpage>
<pub-id pub-id-type="doi">10.1889/1.2785492</pub-id>
</mixed-citation>
</ref>
<ref id="B9"><mixed-citation publication-type="journal"><name><surname>Simons</surname>
<given-names>DJ</given-names>
</name>
<article-title>Attentional capture and inattentional blindness</article-title>
<source>Trends in Cognitive Sciences</source>
<year>2000</year>
<volume>4</volume>
<fpage>147</fpage>
<lpage>155</lpage>
<pub-id pub-id-type="doi">10.1016/S1364-6613(00)01455-8</pub-id>
<pub-id pub-id-type="pmid">10740279</pub-id>
</mixed-citation>
</ref>
<ref id="B10"><mixed-citation publication-type="book"><name><surname>Suh</surname>
<given-names>B</given-names>
</name>
<name><surname>Ling</surname>
<given-names>H</given-names>
</name>
<name><surname>Bederson</surname>
<given-names>BB</given-names>
</name>
<name><surname>Jacobs</surname>
<given-names>DW</given-names>
</name>
<article-title>Automatic thumbnail cropping and its effectiveness</article-title>
<source>Proceedings of the 16th annual ACM symposium on User interface software and technology</source>
<year>2003</year>
<publisher-name>Vancouver, Canada: ACM</publisher-name>
<fpage>95</fpage>
<lpage>104</lpage>
<comment>full_text</comment>
</mixed-citation>
</ref>
<ref id="B11"><mixed-citation publication-type="journal"><name><surname>Chen</surname>
<given-names>LQ</given-names>
</name>
<name><surname>Xie</surname>
<given-names>X</given-names>
</name>
<name><surname>Fan</surname>
<given-names>X</given-names>
</name>
<name><surname>Ma</surname>
<given-names>WY</given-names>
</name>
<name><surname>Zhang</surname>
<given-names>HJ</given-names>
</name>
<name><surname>Zhou</surname>
<given-names>HQ</given-names>
</name>
<article-title>A visual attention model for adapting images on small displays</article-title>
<source>Multimedia Systems</source>
<year>2003</year>
<volume>9</volume>
<fpage>353</fpage>
<lpage>364</lpage>
<pub-id pub-id-type="doi">10.1007/s00530-003-0105-4</pub-id>
</mixed-citation>
</ref>
<ref id="B12"><mixed-citation publication-type="book"><name><surname>Liu</surname>
<given-names>F</given-names>
</name>
<name><surname>Gleicher</surname>
<given-names>M</given-names>
</name>
<article-title>Automatic image retargeting with fisheye-view warping</article-title>
<source>Proceedings of the 18th annual ACM symposium on User interface software and technology</source>
<year>2005</year>
<publisher-name>Seattle, WA, USA: ACM</publisher-name>
<fpage>153</fpage>
<lpage>162</lpage>
<comment>full_text</comment>
</mixed-citation>
</ref>
<ref id="B13"><mixed-citation publication-type="book"><name><surname>Setlur</surname>
<given-names>V</given-names>
</name>
<name><surname>Takagi</surname>
<given-names>S</given-names>
</name>
<name><surname>Raskar</surname>
<given-names>R</given-names>
</name>
<name><surname>Gleicher</surname>
<given-names>M</given-names>
</name>
<name><surname>Gooch</surname>
<given-names>B</given-names>
</name>
<article-title>Automatic image retargeting</article-title>
<source>Proceedings of the 4th international conference on Mobile and ubiquitous multimedia</source>
<year>2005</year>
<volume>154</volume>
<publisher-name>Christchurch, New Zealand: ACM</publisher-name>
<fpage>59</fpage>
<lpage>68</lpage>
<comment>full_text</comment>
</mixed-citation>
</ref>
<ref id="B14"><mixed-citation publication-type="journal"><name><surname>Setlur</surname>
<given-names>V</given-names>
</name>
<name><surname>Lechner</surname>
<given-names>T</given-names>
</name>
<name><surname>Nienhaus</surname>
<given-names>M</given-names>
</name>
<name><surname>Gooch</surname>
<given-names>B</given-names>
</name>
<article-title>Retargeting images and video for preserving information saliency</article-title>
<source>IEEE Computer Graphics and Applications</source>
<year>2007</year>
<volume>27</volume>
<fpage>80</fpage>
<lpage>88</lpage>
<pub-id pub-id-type="doi">10.1109/MCG.2007.133</pub-id>
<pub-id pub-id-type="pmid">17913027</pub-id>
</mixed-citation>
</ref>
<ref id="B15"><mixed-citation publication-type="journal"><name><surname>Avidan</surname>
<given-names>S</given-names>
</name>
<name><surname>Shamir</surname>
<given-names>A</given-names>
</name>
<article-title>Seam carving for content-aware image resizing</article-title>
<source>ACM Transactions on Graphics</source>
<year>2007</year>
<volume>26</volume>
<fpage>3</fpage>
<pub-id pub-id-type="doi">10.1145/1276377.1276390</pub-id>
</mixed-citation>
</ref>
<ref id="B16"><mixed-citation publication-type="journal"><name><surname>Rubinstein</surname>
<given-names>M</given-names>
</name>
<name><surname>Shamir</surname>
<given-names>A</given-names>
</name>
<name><surname>Avidan</surname>
<given-names>S</given-names>
</name>
<article-title>Improved seam carving for video retargeting</article-title>
<source>ACM Transactions on Graphics</source>
<year>2008</year>
<volume>27</volume>
<issue>3</issue>
<fpage>1</fpage>
<lpage>9</lpage>
<pub-id pub-id-type="doi">10.1145/1360612.1360615</pub-id>
</mixed-citation>
</ref>
<ref id="B17"><mixed-citation publication-type="journal"><name><surname>Wolf</surname>
<given-names>L</given-names>
</name>
<name><surname>Guttmann</surname>
<given-names>M</given-names>
</name>
<name><surname>Cohen-Or</surname>
<given-names>D</given-names>
</name>
<article-title>Non-homogeneous content-driven video-retargeting</article-title>
<source>2007 IEEE 11th International Conference on Computer Vision</source>
<year>2007</year>
<volume>1-6</volume>
<fpage>1418</fpage>
<lpage>1423</lpage>
</mixed-citation>
</ref>
<ref id="B18"><mixed-citation publication-type="journal"><name><surname>Zhang</surname>
<given-names>YF</given-names>
</name>
<name><surname>Hu</surname>
<given-names>SM</given-names>
</name>
<name><surname>Martin</surname>
<given-names>RR</given-names>
</name>
<article-title>Shrinkability Maps for Content-Aware Video Resizing</article-title>
<source>Computer Graphics Forum</source>
<year>2008</year>
<volume>27</volume>
<fpage>1797</fpage>
<lpage>1804</lpage>
<pub-id pub-id-type="doi">10.1111/j.1467-8659.2008.01325.x</pub-id>
</mixed-citation>
</ref>
<ref id="B19"><mixed-citation publication-type="other"><name><surname>Kim</surname>
<given-names>J-S</given-names>
</name>
<name><surname>Kim</surname>
<given-names>J-H</given-names>
</name>
<name><surname>Kim</surname>
<given-names>C-S</given-names>
</name>
<article-title>Adaptive image and video retargeting technique based on Fourier analysis</article-title>
<source>Proc IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops CVPR Workshops</source>
<year>2009</year>
<fpage>1730</fpage>
<lpage>1737</lpage>
</mixed-citation>
</ref>
<ref id="B20"><mixed-citation publication-type="journal"><name><surname>Fleck</surname>
<given-names>MM</given-names>
</name>
<article-title>Some defects in finite-difference edge finders</article-title>
<source>#IEEE_J_PAMI#</source>
<year>1992</year>
<volume>14</volume>
<fpage>337</fpage>
<lpage>345</lpage>
</mixed-citation>
</ref>
<ref id="B21"><mixed-citation publication-type="journal"><name><surname>Canny</surname>
<given-names>J</given-names>
</name>
<article-title>A Computational Approach to Edge Detection</article-title>
<source>#IEEE_J_PAMI#</source>
<year>1986</year>
<volume>PAMI-8</volume>
<fpage>679</fpage>
<lpage>698</lpage>
</mixed-citation>
</ref>
<ref id="B22"><mixed-citation publication-type="journal"><name><surname>Jobson</surname>
<given-names>DJ</given-names>
</name>
<name><surname>Rahman</surname>
<given-names>Z</given-names>
</name>
<name><surname>Woodell</surname>
<given-names>GA</given-names>
</name>
<article-title>A multiscale retinex for bridging the gap between color images and the human observation of scenes</article-title>
<source>#IEEE_J_IP#</source>
<year>1997</year>
<volume>6</volume>
<fpage>965</fpage>
<lpage>976</lpage>
</mixed-citation>
</ref>
<ref id="B23"><mixed-citation publication-type="journal"><name><surname>Liu</surname>
<given-names>SC</given-names>
</name>
<name><surname>Fu</surname>
<given-names>CW</given-names>
</name>
<name><surname>Chang</surname>
<given-names>SY</given-names>
</name>
<article-title>Statistical change detection with moments under time-varying illumination</article-title>
<source>IEEE Transactions on Image Processing</source>
<year>1998</year>
<volume>7</volume>
<fpage>1258</fpage>
<lpage>1268</lpage>
<pub-id pub-id-type="doi">10.1109/83.709658</pub-id>
<pub-id pub-id-type="pmid">18276338</pub-id>
</mixed-citation>
</ref>
<ref id="B24"><mixed-citation publication-type="book"><name><surname>Bovik</surname>
<given-names>AC</given-names>
</name>
<source>Handbook of Image and Video Processing</source>
<year>2005</year>
<publisher-name>Orlando, FL, USA: Academic Press, Inc</publisher-name>
</mixed-citation>
</ref>
<ref id="B25"><mixed-citation publication-type="journal"><name><surname>Fant</surname>
<given-names>KM</given-names>
</name>
<article-title>A Nonaliasing, Real-Time Spatial Transform Technique</article-title>
<source>IEEE Computer Graphics and Applications</source>
<year>1986</year>
<volume>6</volume>
<fpage>71</fpage>
<lpage>80</lpage>
<pub-id pub-id-type="doi">10.1109/MCG.1986.276613</pub-id>
</mixed-citation>
</ref>
<ref id="B26"><mixed-citation publication-type="journal"><name><surname>Spitz</surname>
<given-names>AL</given-names>
</name>
<article-title>Shape-based word recognition</article-title>
<source>International Journal on Document Analysis and Recognition</source>
<year>1999</year>
<volume>1</volume>
<fpage>178</fpage>
<lpage>190</lpage>
<pub-id pub-id-type="doi">10.1007/s100320050017</pub-id>
</mixed-citation>
</ref>
<ref id="B27"><mixed-citation publication-type="journal"><name><surname>Degenaar</surname>
<given-names>P</given-names>
</name>
<name><surname>Grossman</surname>
<given-names>N</given-names>
</name>
<name><surname>Memon</surname>
<given-names>MA</given-names>
</name>
<name><surname>Burrone</surname>
<given-names>J</given-names>
</name>
<name><surname>Dawson</surname>
<given-names>M</given-names>
</name>
<name><surname>Drakakis</surname>
<given-names>E</given-names>
</name>
<name><surname>Neil</surname>
<given-names>M</given-names>
</name>
<name><surname>Nikolic</surname>
<given-names>K</given-names>
</name>
<article-title>Optobionic vision-a new genetically enhanced light on retinal prosthesis</article-title>
<source>Journal of Neural Engineering</source>
<year>2009</year>
<volume>6</volume>
<fpage>035007</fpage>
<pub-id pub-id-type="doi">10.1088/1741-2560/6/3/035007</pub-id>
<pub-id pub-id-type="pmid">19458396</pub-id>
</mixed-citation>
</ref>
<ref id="B28"><mixed-citation publication-type="journal"><name><surname>Al-Atabany</surname>
<given-names>W</given-names>
</name>
<name><surname>Memon</surname>
<given-names>M</given-names>
</name>
<name><surname>Downes</surname>
<given-names>S</given-names>
</name>
<name><surname>Degenaar</surname>
<given-names>P</given-names>
</name>
<article-title>Designing and testing scene enhancement algorithms for patients with retina degenerative disorders</article-title>
<source>BioMedical Engineering OnLine</source>
<year>2010</year>
<volume>9</volume>
<fpage>27</fpage>
<pub-id pub-id-type="doi">10.1186/1475-925X-9-27</pub-id>
<pub-id pub-id-type="pmid">20565870</pub-id>
</mixed-citation>
</ref>
<ref id="B29"><mixed-citation publication-type="other"><name><surname>Atabany</surname>
<given-names>W</given-names>
</name>
<name><surname>Degenaar</surname>
<given-names>P</given-names>
</name>
<article-title>Parallelism to reduce power consumption on FPGA spatiotemporal image processing</article-title>
<source>Proc IEEE International Symposium on Circuits and Systems ISCAS</source>
<year>2008</year>
<fpage>1476</fpage>
<lpage>1479</lpage>
</mixed-citation>
</ref>
</ref-list>
</back>
</pmc>
<affiliations><list><country><li>Royaume-Uni</li>
<li>Égypte</li>
</country>
<region><li>Angleterre</li>
<li>Grand Londres</li>
</region>
<settlement><li>Londres</li>
</settlement>
</list>
<tree><country name="Royaume-Uni"><region name="Angleterre"><name sortKey="Al Atabany, Walid I" sort="Al Atabany, Walid I" uniqKey="Al Atabany W" first="Walid I" last="Al-Atabany">Walid I. Al-Atabany</name>
</region>
<name sortKey="Degenaar, Patrick A" sort="Degenaar, Patrick A" uniqKey="Degenaar P" first="Patrick A" last="Degenaar">Patrick A. Degenaar</name>
<name sortKey="Tong, Tzyy" sort="Tong, Tzyy" uniqKey="Tong T" first="Tzyy" last="Tong">Tzyy Tong</name>
</country>
<country name="Égypte"><noRegion><name sortKey="Al Atabany, Walid I" sort="Al Atabany, Walid I" uniqKey="Al Atabany W" first="Walid I" last="Al-Atabany">Walid I. Al-Atabany</name>
</noRegion>
</country>
</tree>
</affiliations>
</record>
Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Pmc/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000146 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd -nk 000146 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien |wiki= Ticri/CIDE |area= OcrV1 |flux= Pmc |étape= Checkpoint |type= RBID |clé= PMC:2949883 |texte= Improved content aware scene retargeting for retinitis pigmentosa patients }}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/RBID.i -Sk "pubmed:20846440" \ | HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd \ | NlmPubMed2Wicri -a OcrV1
![]() | This area was generated with Dilib version V0.6.32. | ![]() |