Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?
Identifieur interne : 000731 ( Pmc/Checkpoint ); précédent : 000730; suivant : 000732Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?
Auteurs : David J. Brown [Royaume-Uni] ; Andrew J. R. Simpson [Royaume-Uni] ; Michael J. Proulx [Royaume-Uni]Source :
- Frontiers in Psychology [ 1664-1078 ] ; 2015.
Abstract
A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don’t yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (
Url:
DOI: 10.3389/fpsyg.2015.01522
PubMed: 26528202
PubMed Central: 4602098
Affiliations:
Links toward previous steps (curation, corpus...)
Links to Exploration step
PMC:4602098Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?</title><author><name sortKey="Brown, David J" sort="Brown, David J" uniqKey="Brown D" first="David J." last="Brown">David J. Brown</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Crossmodal Cognition Lab, Department of Psychology, University of Bath</institution><country>Bath, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><institution>Biological and Experimental Psychology Group, School of Biological and Chemical Sciences, Queen Mary University of London</institution><country>London, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Simpson, Andrew J R" sort="Simpson, Andrew J R" uniqKey="Simpson A" first="Andrew J. R." last="Simpson">Andrew J. R. Simpson</name><affiliation wicri:level="1"><nlm:aff id="aff3"><institution>Centre for Vision, Speech and Signal Processing, University of Surrey</institution><country>Guildford, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Proulx, Michael J" sort="Proulx, Michael J" uniqKey="Proulx M" first="Michael J." last="Proulx">Michael J. Proulx</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Crossmodal Cognition Lab, Department of Psychology, University of Bath</institution><country>Bath, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">26528202</idno><idno type="pmc">4602098</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC4602098</idno><idno type="RBID">PMC:4602098</idno><idno type="doi">10.3389/fpsyg.2015.01522</idno><date when="2015">2015</date><idno type="wicri:Area/Pmc/Corpus">000214</idno><idno type="wicri:Area/Pmc/Curation">000214</idno><idno type="wicri:Area/Pmc/Checkpoint">000731</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?</title><author><name sortKey="Brown, David J" sort="Brown, David J" uniqKey="Brown D" first="David J." last="Brown">David J. Brown</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Crossmodal Cognition Lab, Department of Psychology, University of Bath</institution><country>Bath, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><institution>Biological and Experimental Psychology Group, School of Biological and Chemical Sciences, Queen Mary University of London</institution><country>London, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Simpson, Andrew J R" sort="Simpson, Andrew J R" uniqKey="Simpson A" first="Andrew J. R." last="Simpson">Andrew J. R. Simpson</name><affiliation wicri:level="1"><nlm:aff id="aff3"><institution>Centre for Vision, Speech and Signal Processing, University of Surrey</institution><country>Guildford, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author><author><name sortKey="Proulx, Michael J" sort="Proulx, Michael J" uniqKey="Proulx M" first="Michael J." last="Proulx">Michael J. Proulx</name><affiliation wicri:level="1"><nlm:aff id="aff1"><institution>Crossmodal Cognition Lab, Department of Psychology, University of Bath</institution><country>Bath, UK</country></nlm:aff><country xml:lang="fr">Royaume-Uni</country><wicri:regionArea></wicri:regionArea><wicri:regionArea># see nlm:aff region in country</wicri:regionArea></affiliation></author></analytic><series><title level="j">Frontiers in Psychology</title><idno type="eISSN">1664-1078</idno><imprint><date when="2015">2015</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don’t yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (<italic>N</italic> = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio–visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Abboud, S" uniqKey="Abboud S">S. Abboud</name></author><author><name sortKey="Hanassy, S" uniqKey="Hanassy S">S. Hanassy</name></author><author><name sortKey="Levy Tzedek, S" uniqKey="Levy Tzedek S">S. Levy-Tzedek</name></author><author><name sortKey="Maidenbaum, S" uniqKey="Maidenbaum S">S. Maidenbaum</name></author><author><name sortKey="Amedi, A" uniqKey="Amedi A">A. Amedi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Amedi, A" uniqKey="Amedi A">A. Amedi</name></author><author><name sortKey="Stern, W M" uniqKey="Stern W">W. M. Stern</name></author><author><name sortKey="Camprodon, J A" uniqKey="Camprodon J">J. A. Camprodon</name></author><author><name sortKey="Bermpohl, F" uniqKey="Bermpohl F">F. Bermpohl</name></author><author><name sortKey="Merabet, L" uniqKey="Merabet L">L. Merabet</name></author><author><name sortKey="Rotman, S" uniqKey="Rotman S">S. Rotman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Arehart, K H" uniqKey="Arehart K">K. H. Arehart</name></author><author><name sortKey="Rosengard, P F" uniqKey="Rosengard P">P. F. Rosengard</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Auvray, M" uniqKey="Auvray M">M. Auvray</name></author><author><name sortKey="Hanneton, S" uniqKey="Hanneton S">S. Hanneton</name></author><author><name sortKey="O Egan, J K" uniqKey="O Egan J">J. K. O’Regan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bach Y Rita, P" uniqKey="Bach Y Rita P">P. Bach-y-Rita</name></author><author><name sortKey="Kercel, S W" uniqKey="Kercel S">S. W. Kercel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ben Artzi, E" uniqKey="Ben Artzi E">E. Ben-Artzi</name></author><author><name sortKey="Marks, L E" uniqKey="Marks L">L. E. Marks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ben Av, M B" uniqKey="Ben Av M">M. B. Ben-Av</name></author><author><name sortKey="Sagi, D" uniqKey="Sagi D">D. Sagi</name></author><author><name sortKey="Braun, J" uniqKey="Braun J">J. Braun</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bermant, R I" uniqKey="Bermant R">R. I. Bermant</name></author><author><name sortKey="Welch, R B" uniqKey="Welch R">R. B. Welch</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bernstein, I H" uniqKey="Bernstein I">I. H. Bernstein</name></author><author><name sortKey="Edelstein, B A" uniqKey="Edelstein B">B. A. Edelstein</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bertelson, P" uniqKey="Bertelson P">P. Bertelson</name></author><author><name sortKey="Radeau, M" uniqKey="Radeau M">M. Radeau</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bregman, A S" uniqKey="Bregman A">A. S. Bregman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bregman, A S" uniqKey="Bregman A">A. S. Bregman</name></author><author><name sortKey="Liao, C" uniqKey="Liao C">C. Liao</name></author><author><name sortKey="Levitan, R" uniqKey="Levitan R">R. Levitan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Brown, D J" uniqKey="Brown D">D. J. Brown</name></author><author><name sortKey="Macpherson, T" uniqKey="Macpherson T">T. Macpherson</name></author><author><name sortKey="Ward, J" uniqKey="Ward J">J. Ward</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Brown, D J" uniqKey="Brown D">D. J. Brown</name></author><author><name sortKey="Simpson, A J R" uniqKey="Simpson A">A. J. R. Simpson</name></author><author><name sortKey="Proulx, M J" uniqKey="Proulx M">M. J. Proulx</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Cherry, C" uniqKey="Cherry C">C. Cherry</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Collignon, O" uniqKey="Collignon O">O. Collignon</name></author><author><name sortKey="Lassonde, M" uniqKey="Lassonde M">M. Lassonde</name></author><author><name sortKey="Lepore, F" uniqKey="Lepore F">F. Lepore</name></author><author><name sortKey="Bastien, D" uniqKey="Bastien D">D. Bastien</name></author><author><name sortKey="Veraart, C" uniqKey="Veraart C">C. Veraart</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Driver, J" uniqKey="Driver J">J. Driver</name></author><author><name sortKey="Baylis, G C" uniqKey="Baylis G">G. C. Baylis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Evans, K K" uniqKey="Evans K">K. K. Evans</name></author><author><name sortKey="Treisman, A" uniqKey="Treisman A">A. Treisman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fendrich, R" uniqKey="Fendrich R">R. Fendrich</name></author><author><name sortKey="Corballis, P M" uniqKey="Corballis P">P. M. Corballis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Frasnelli, J" uniqKey="Frasnelli J">J. Frasnelli</name></author><author><name sortKey="Collignon, O" uniqKey="Collignon O">O. Collignon</name></author><author><name sortKey="Voss, P" uniqKey="Voss P">P. Voss</name></author><author><name sortKey="Lepore, F" uniqKey="Lepore F">F. Lepore</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Frassinetti, F" uniqKey="Frassinetti F">F. Frassinetti</name></author><author><name sortKey="Bolognini, N" uniqKey="Bolognini N">N. Bolognini</name></author><author><name sortKey="Ladavas, E" uniqKey="Ladavas E">E. Ladavas</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Frauenberger, C" uniqKey="Frauenberger C">C. Frauenberger</name></author><author><name sortKey="Stockman, T" uniqKey="Stockman T">T. Stockman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Haigh, A" uniqKey="Haigh A">A. Haigh</name></author><author><name sortKey="Brown, D J" uniqKey="Brown D">D. J. Brown</name></author><author><name sortKey="Meijer, P" uniqKey="Meijer P">P. Meijer</name></author><author><name sortKey="Proulx, M J" uniqKey="Proulx M">M. J. Proulx</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Iordanescu, L" uniqKey="Iordanescu L">L. Iordanescu</name></author><author><name sortKey="Guzman Martinez, E" uniqKey="Guzman Martinez E">E. Guzman-Martinez</name></author><author><name sortKey="Grabowecky, M" uniqKey="Grabowecky M">M. Grabowecky</name></author><author><name sortKey="Suzuki, S" uniqKey="Suzuki S">S. Suzuki</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Jimenez, J" uniqKey="Jimenez J">J. Jiménez</name></author><author><name sortKey="Olea, J" uniqKey="Olea J">J. Olea</name></author><author><name sortKey="Torres, J" uniqKey="Torres J">J. Torres</name></author><author><name sortKey="Alonso, I" uniqKey="Alonso I">I. Alonso</name></author><author><name sortKey="Harder, D" uniqKey="Harder D">D. Harder</name></author><author><name sortKey="Fischer, K" uniqKey="Fischer K">K. Fischer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kleczkowski, P" uniqKey="Kleczkowski P">P. Kleczkowski</name></author><author><name sortKey="Pluta, M" uniqKey="Pluta M">M. Pluta</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Laurienti, P J" uniqKey="Laurienti P">P. J. Laurienti</name></author><author><name sortKey="Kraft, R A" uniqKey="Kraft R">R. A. Kraft</name></author><author><name sortKey="Maldjian, J A" uniqKey="Maldjian J">J. A. Maldjian</name></author><author><name sortKey="Burdette, J H" uniqKey="Burdette J">J. H. Burdette</name></author><author><name sortKey="Wallace, M T" uniqKey="Wallace M">M. T. Wallace</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Macdonald, D" uniqKey="Macdonald D">D. MacDonald</name></author><author><name sortKey="Stockman, T" uniqKey="Stockman T">T. Stockman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Maidenbaum, S" uniqKey="Maidenbaum S">S. Maidenbaum</name></author><author><name sortKey="Levy Tzedek, S" uniqKey="Levy Tzedek S">S. Levy-Tzedek</name></author><author><name sortKey="Chebat, D R" uniqKey="Chebat D">D. R. Chebat</name></author><author><name sortKey="Amedi, A" uniqKey="Amedi A">A. Amedi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Marks, L E" uniqKey="Marks L">L. E. Marks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Meijer, P" uniqKey="Meijer P">P. Meijer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pascual Leone, A" uniqKey="Pascual Leone A">A. Pascual-Leone</name></author><author><name sortKey="Hamilton, R" uniqKey="Hamilton R">R. Hamilton</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Plomp, R" uniqKey="Plomp R">R. Plomp</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Plomp, R" uniqKey="Plomp R">R. Plomp</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Plomp, R" uniqKey="Plomp R">R. Plomp</name></author><author><name sortKey="Levelt, W J M" uniqKey="Levelt W">W. J. M. Levelt</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Proulx, M J" uniqKey="Proulx M">M. J. Proulx</name></author><author><name sortKey="Brown, D J" uniqKey="Brown D">D. J. Brown</name></author><author><name sortKey="Pasqualotto, A" uniqKey="Pasqualotto A">A. Pasqualotto</name></author><author><name sortKey="Meijer, P" uniqKey="Meijer P">P. Meijer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Proulx, M J" uniqKey="Proulx M">M. J. Proulx</name></author><author><name sortKey="Stoerig, P" uniqKey="Stoerig P">P. Stoerig</name></author><author><name sortKey="Ludowig, E" uniqKey="Ludowig E">E. Ludowig</name></author><author><name sortKey="Knoll, I" uniqKey="Knoll I">I. Knoll</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Radeau, M" uniqKey="Radeau M">M. Radeau</name></author><author><name sortKey="Bertelson, P" uniqKey="Bertelson P">P. Bertelson</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Recanzone, G H" uniqKey="Recanzone G">G. H. Recanzone</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Renier, L" uniqKey="Renier L">L. Renier</name></author><author><name sortKey="Collignon, O" uniqKey="Collignon O">O. Collignon</name></author><author><name sortKey="Poirier, C" uniqKey="Poirier C">C. Poirier</name></author><author><name sortKey="Tranduy, D" uniqKey="Tranduy D">D. Tranduy</name></author><author><name sortKey="Vanlierde, A" uniqKey="Vanlierde A">A. Vanlierde</name></author><author><name sortKey="Bol, A" uniqKey="Bol A">A. Bol</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Roder, B" uniqKey="Roder B">B. Roder</name></author><author><name sortKey="Teder Salejarvi, W" uniqKey="Teder Salejarvi W">W. Teder-Salejarvi</name></author><author><name sortKey="Sterr, A" uniqKey="Sterr A">A. Sterr</name></author><author><name sortKey="Rosler, F" uniqKey="Rosler F">F. Rosler</name></author><author><name sortKey="Hillyard, S A" uniqKey="Hillyard S">S. A. Hillyard</name></author><author><name sortKey="Neville, H J" uniqKey="Neville H">H. J. Neville</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rowell, J" uniqKey="Rowell J">J. Rowell</name></author><author><name sortKey="Ungar, S" uniqKey="Ungar S">S. Ungar</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sharpee, T O" uniqKey="Sharpee T">T. O. Sharpee</name></author><author><name sortKey="Atencio, C A" uniqKey="Atencio C">C. A. Atencio</name></author><author><name sortKey="Schreiner, C E" uniqKey="Schreiner C">C. E. Schreiner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Shower, E" uniqKey="Shower E">E. Shower</name></author><author><name sortKey="Biddulph, R" uniqKey="Biddulph R">R. Biddulph</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Soto Faraco, S" uniqKey="Soto Faraco S">S. Soto-Faraco</name></author><author><name sortKey="Lyons, J" uniqKey="Lyons J">J. Lyons</name></author><author><name sortKey="Gazzaniga, M" uniqKey="Gazzaniga M">M. Gazzaniga</name></author><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author><author><name sortKey="Kingstone, A" uniqKey="Kingstone A">A. Kingstone</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Soto Faraco, S" uniqKey="Soto Faraco S">S. Soto-Faraco</name></author><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author><author><name sortKey="Kingstone, A" uniqKey="Kingstone A">A. Kingstone</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Spence, C" uniqKey="Spence C">C. Spence</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Striem Amit, E" uniqKey="Striem Amit E">E. Striem-Amit</name></author><author><name sortKey="Amedi, A" uniqKey="Amedi A">A. Amedi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Striem Amit, E" uniqKey="Striem Amit E">E. Striem-Amit</name></author><author><name sortKey="Guendelman, M" uniqKey="Guendelman M">M. Guendelman</name></author><author><name sortKey="Amedi, A" uniqKey="Amedi A">A. Amedi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Thurlow, W R" uniqKey="Thurlow W">W. R. Thurlow</name></author><author><name sortKey="Bernstein, S" uniqKey="Bernstein S">S. Bernstein</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vroomen, J" uniqKey="Vroomen J">J. Vroomen</name></author><author><name sortKey="De Gelder, B" uniqKey="De Gelder B">B. de Gelder</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wan, C Y" uniqKey="Wan C">C. Y. Wan</name></author><author><name sortKey="Wood, A G" uniqKey="Wood A">A. G. Wood</name></author><author><name sortKey="Reutens, D C" uniqKey="Reutens D">D. C. Reutens</name></author><author><name sortKey="Wilson, S J" uniqKey="Wilson S">S. J. Wilson</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ward, J" uniqKey="Ward J">J. Ward</name></author><author><name sortKey="Meijer, P" uniqKey="Meijer P">P. Meijer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wever, E" uniqKey="Wever E">E. Wever</name></author><author><name sortKey="Wedell, C" uniqKey="Wedell C">C. Wedell</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Front Psychol</journal-id><journal-id journal-id-type="iso-abbrev">Front Psychol</journal-id><journal-id journal-id-type="publisher-id">Front. Psychol.</journal-id><journal-title-group><journal-title>Frontiers in Psychology</journal-title></journal-title-group><issn pub-type="epub">1664-1078</issn><publisher><publisher-name>Frontiers Media S.A.</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">26528202</article-id><article-id pub-id-type="pmc">4602098</article-id><article-id pub-id-type="doi">10.3389/fpsyg.2015.01522</article-id><article-categories><subj-group subj-group-type="heading"><subject>Psychology</subject><subj-group><subject>Original Research</subject></subj-group></subj-group></article-categories><title-group><article-title>Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Brown</surname><given-names>David J.</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="aff" rid="aff2"><sup>2</sup></xref><xref ref-type="author-notes" rid="fn001"><sup>*</sup></xref></contrib><contrib contrib-type="author"><name><surname>Simpson</surname><given-names>Andrew J. R.</given-names></name><xref ref-type="aff" rid="aff3"><sup>3</sup></xref><uri xlink:type="simple" xlink:href="http://loop.frontiersin.org/people/126956/overview"></uri></contrib><contrib contrib-type="author"><name><surname>Proulx</surname><given-names>Michael J.</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="author-notes" rid="fn001"><sup>*</sup></xref><uri xlink:type="simple" xlink:href="http://loop.frontiersin.org/people/34273/overview"></uri></contrib></contrib-group><aff id="aff1"><sup>1</sup><institution>Crossmodal Cognition Lab, Department of Psychology, University of Bath</institution><country>Bath, UK</country></aff><aff id="aff2"><sup>2</sup><institution>Biological and Experimental Psychology Group, School of Biological and Chemical Sciences, Queen Mary University of London</institution><country>London, UK</country></aff><aff id="aff3"><sup>3</sup><institution>Centre for Vision, Speech and Signal Processing, University of Surrey</institution><country>Guildford, UK</country></aff><author-notes><fn fn-type="edited-by"><p>Edited by: <italic>Achille Pasqualotto, Sabanci University, Turkey</italic></p></fn><fn fn-type="edited-by"><p>Reviewed by: <italic>Pietro Pietrini, Azienda Ospedaliero-Universitaria Pisana, Italy; Tina Iachini, Second University of Naples, Italy</italic></p></fn><corresp id="fn001">*Correspondence: <italic>David J. Brown and Michael J. Proulx, Crossmodal Cognition Lab, Department of Psychology, University of Bath, 2 South, Bath, BA2 7AY, UK, <email xlink:type="simple">djbrownmsp@gmail.com</email>; <email xlink:type="simple">m.j.proulx@bath.ac.uk</email></italic></corresp><fn fn-type="other" id="fn002"><p>This article was submitted to Cognition, a section of the journal Frontiers in Psychology</p></fn></author-notes><pub-date pub-type="epub"><day>13</day><month>10</month><year>2015</year></pub-date><pub-date pub-type="collection"><year>2015</year></pub-date><volume>6</volume><elocation-id>1522</elocation-id><history><date date-type="received"><day>13</day><month>4</month><year>2015</year></date><date date-type="accepted"><day>22</day><month>9</month><year>2015</year></date></history><permissions><copyright-statement>Copyright © 2015 Brown, Simpson and Proulx.</copyright-statement><copyright-year>2015</copyright-year><copyright-holder>Brown, Simpson and Proulx</copyright-holder><license xlink:href="http://creativecommons.org/licenses/by/4.0/"><license-p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) or licensor are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.</license-p></license></permissions><abstract><p>A critical task for the brain is the sensory representation and identification of perceptual objects in the world. When the visual sense is impaired, hearing and touch must take primary roles and in recent times compensatory techniques have been developed that employ the tactile or auditory system as a substitute for the visual system. Visual-to-auditory sonifications provide a complex, feature-based auditory representation that must be decoded and integrated into an object-based representation by the listener. However, we don’t yet know what role the auditory system plays in the object integration stage and whether the principles of auditory scene analysis apply. Here we used coarse sonified images in a two-tone discrimination task to test whether auditory feature-based representations of visual objects would be confounded when their features conflicted with the principles of auditory consonance. We found that listeners (<italic>N</italic> = 36) performed worse in an object recognition task when the auditory feature-based representation was harmonically consonant. We also found that this conflict was not negated with the provision of congruent audio–visual information. The findings suggest that early auditory processes of harmonic grouping dominate the object formation process and that the complexity of the signal, and additional sensory information have limited effect on this.</p></abstract><kwd-group><kwd>auditory scene analysis</kwd><kwd>consonance</kwd><kwd>signal complexity</kwd><kwd>blindness</kwd><kwd>cross-modal</kwd><kwd>sensory substitution</kwd></kwd-group><funding-group><award-group><funding-source id="cn001">EPSRC<named-content content-type="fundref-id">10.13039/501100000266</named-content></funding-source><award-id rid="cn001">EP/J017205/1</award-id></award-group></funding-group><counts><fig-count count="5"></fig-count><table-count count="1"></table-count><equation-count count="0"></equation-count><ref-count count="54"></ref-count><page-count count="9"></page-count><word-count count="0"></word-count></counts></article-meta></front><body><sec><title>Introduction</title><p>Our sensory systems provide a rich coherent representation of the world through the integration and discrimination of input from multiple sensory modalities (<xref rid="B47" ref-type="bibr">Spence, 2011</xref>). These low-level processes are modulated by high-order processing to selectively attend to task relevant stimuli. For example to attend to a speaker at a cocktail party we must select the low-level acoustic features that are relevant to the target, that is the person you are speaking with, from the environmental noise (<xref rid="B15" ref-type="bibr">Cherry, 1953</xref>). To accomplish this, feature-based sensory representations must be recombined into object-based representations in a rule based manner. In visual perception this is through scene analysis. Visual input is grouped into distinct objects based on Gestalt grouping rules such as feature proximity, similarity, continuity, closure, figure ground, and common fate (<xref rid="B17" ref-type="bibr">Driver and Baylis, 1989</xref>; <xref rid="B7" ref-type="bibr">Ben-Av et al., 1992</xref>). Similarly, there are rules that govern the arrangement of low-level stimuli into haptic and auditory objects. For the latter the process is called auditory scene analysis (ASA). Contrary to the spatial principles that guide visual categorization, grouping in ASA is at either a temporal or melodic level governed by proximity or similarity over time, pitch or loudness continuation, or at spectral levels including common fate, coherent changes in loudness, frequency, or harmony (<xref rid="B11" ref-type="bibr">Bregman, 1994</xref>).</p><p>While principles of ASA, such as frequency and harmony, may seem relatively unimportant to visual perception they hold relevance for rehabilitation techniques for the substitution of vision for the visually impaired (<xref rid="B37" ref-type="bibr">Proulx et al., 2008</xref>; <xref rid="B13" ref-type="bibr">Brown et al., 2011</xref>). Researchers have long strived to provide crucial visual information with compensatory techniques via alternate modalities such as touch – Braille, embossed maps, tactile sensory substitution – (<xref rid="B5" ref-type="bibr">Bach-y-Rita and Kercel, 2003</xref>; <xref rid="B42" ref-type="bibr">Rowell and Ungar, 2003</xref>; <xref rid="B25" ref-type="bibr">Jiménez et al., 2009</xref>) or more recently sound – auditory sensory substitution and auditory workspaces – (<xref rid="B22" ref-type="bibr">Frauenberger and Stockman, 2009</xref>; <xref rid="B1" ref-type="bibr">Abboud et al., 2014</xref>; <xref rid="B28" ref-type="bibr">MacDonald and Stockman, 2014</xref>). The conversion principles of sonification algorithms are not arbitrary but instead based on natural cross-modal correspondences and cross-modal plasticity (<xref rid="B20" ref-type="bibr">Frasnelli et al., 2011</xref>; <xref rid="B47" ref-type="bibr">Spence, 2011</xref>) which allow the coding of visual features (brightness, spatial location) into auditory ones (pitch, loudness, stereo pan). Sensory substitution devices go beyond simple feature detection, and are also effective in ‘visual’ tasks such as object recognition and localisation, and navigation (<xref rid="B4" ref-type="bibr">Auvray et al., 2007</xref>; <xref rid="B13" ref-type="bibr">Brown et al., 2011</xref>; <xref rid="B29" ref-type="bibr">Maidenbaum et al., 2013</xref>). Given that the substitution of vision by other sensory modalities can evoke activity in visual cortex (<xref rid="B40" ref-type="bibr">Renier et al., 2005</xref>; <xref rid="B2" ref-type="bibr">Amedi et al., 2007</xref>; <xref rid="B16" ref-type="bibr">Collignon et al., 2007</xref>), it is unclear whether the mechanisms of scene analysis are processed as visual objects or auditory objects. Is the grouping of feature-based sensory representations into auditory objects based on visual grouping principles or those of ASA?</p><p>It seems natural that if the signal is a sonification it would be processed as an auditory feature and therefore be subjected to grouping principles of ASA. However, with extensive research showing activation of ‘visual’ areas in response to ‘auditory’ stimulation (<xref rid="B2" ref-type="bibr">Amedi et al., 2007</xref>; <xref rid="B48" ref-type="bibr">Striem-Amit and Amedi, 2014</xref>) and visually impaired users defining information from sonifications as ‘visual’ (<xref rid="B53" ref-type="bibr">Ward and Meijer, 2010</xref>) it is important to ascertain whether or not the auditory characteristics are more salient to the final perception using sonifications rather than a straight extrapolation from the unimodal literature. There are certainly valid comparisons between the two modalities. For example, shape and contour are crucial for the organization and recognition of visual objects. In parallel the spectral and temporal contour of a sound, the envelope, is critical in recognizing and organizing auditory objects (<xref rid="B43" ref-type="bibr">Sharpee et al., 2011</xref>).</p><p>However, there are also critical differences. The output signal of the sonification algorithm is dependent of the visual properties of the stimulus and therefore can be a coarse representation relative to a controlled audio-only presentation. For example, the sonification of equal-width visual lines will have different frequency bandwidths dependent on the stimulus baseline on an exponential frequency scale – higher frequency baselines sonify to broader bandwidths, comprise of more sine waves, and are thus more complex than the sonification of an identical line lower down in the visual image. Thus, while the two pieces of visual information are perceived as having equivalent levels of complexity, there is variance between the complexities of the subsequent sonifications. Considering the purpose of sonifications is to convey visual information can we directly apply the principles of ASA, tested using auditory objects, to this?</p><p>If using the analog of two visual lines, equal in length (x-axis) but differing in elevation (y-axis), as two sonifications equal in duration (x-axis) but varying in baseline frequency (y-axis), we can apply ASA to make predictions on the mechanisms of feature segregation. Presented sequentially, with no requirement of identification (the two tones are separated in time), just noticeable differences (JND) in pitch should demonstrate low discrimination thresholds, typically between 1 and 190 Hz dependent on baseline frequency (<xref rid="B44" ref-type="bibr">Shower and Biddulph, 1931</xref>; <xref rid="B54" ref-type="bibr">Wever and Wedell, 1941</xref>). Presented concurrently, discrimination requires the identification of each tone based on the relative frequency components of each object. Considering this is one of the fundamental properties of the ear, the literature on this is scant. <xref rid="B50" ref-type="bibr">Thurlow and Bernstein (1957)</xref> reported two-tone discrimination at around 5% of the baseline frequency (at 4 kHz), while <xref rid="B33" ref-type="bibr">Plomp (1967)</xref>, when assessing the ability to hear a harmonic in a harmonic complex, showed harmonic resolvability for five to seven lower harmonics. <xref rid="B35" ref-type="bibr">Plomp and Levelt (1965)</xref> evaluated explanations of consonance, that is the sensory experience of tonal fusion associated with isolated pairs of tones sharing simple frequency ratios, based on; frequency ratio, harmonic relationships, beats between harmonics, difference tones, and fusion. They concluded that the difference between consonant and dissonant intervals was related to the beats of adjacent partials, and that the transition range between these types of intervals were related to a critical bandwidth.</p><p>While this literature provides a solid grounding to predict results based on ASA it is important to note that in all these experiments the stimuli are generated as auditory objects, often with pure tones. This allows precision of the stimuli based on the exact auditory features you wish to test. For example, pure tones at specific frequencies can be used, or if testing the resolvability of harmonics complexes, tones with exact partials. Within the literature there appear to be no studies that contrast two-tone discrimination in which the precision of the stimuli is not controlled by auditory theory, as would be found when the signal is derived from visual features in a visual-to-auditory sonification. For example, with reference to the two line example above, would interval markers with varying complexity elicit similar results to what is found using controlled auditory stimuli? With this is mind we evaluated the segregation of two ‘auditory’ signals sonified from two equal length parallel lines at varying intervals. In a simple 2AFC paradigm the listener was required to indicate their perception of ‘one-ness’ or ‘two-ness’ in presented tonal complexes(<xref rid="B50" ref-type="bibr">Thurlow and Bernstein, 1957</xref>; <xref rid="B26" ref-type="bibr">Kleczkowski and Pluta, 2012</xref>). Based on the auditory literature we hypothesized that segregation of the two lines into separate objects would be problematic when the sonifications had consonant harmonic relations.</p><p>In a second part of the experiment we used a multisensory paradigm to evaluate whether any influence in discrimination, due to ASA rules, could be negated by the provision of additional information in another modality. Our rationale and methodology were simple. Extensive research has demonstrated the efficacy of using multisensory, rather than uni-modal stimuli, with audio–visual information shown to enhance visual perception (<xref rid="B21" ref-type="bibr">Frassinetti et al., 2002</xref>) visual search (<xref rid="B24" ref-type="bibr">Iordanescu et al., 2008</xref>) and increase performance in spatial and temporal tasks. In speeded classification (SC) paradigms (<xref rid="B18" ref-type="bibr">Evans and Treisman, 2010</xref>) in which participants have to rapidly discriminate visual targets while presented with task irrelevant auditory stimuli, response times increase and accuracy decreases if the auditory stimulus is incongruous, i.e., high visual elevation paired with low pitch tone (<xref rid="B9" ref-type="bibr">Bernstein and Edelstein, 1971</xref>; <xref rid="B30" ref-type="bibr">Marks, 1974</xref>; <xref rid="B6" ref-type="bibr">Ben-Artzi and Marks, 1995</xref>).</p><p>Crucial in multisensory integration is the binding of the unimodal stimuli into one perceived event based on: low-level spatial and temporal synchrony (<xref rid="B47" ref-type="bibr">Spence, 2011</xref>), temporal correlation (<xref rid="B38" ref-type="bibr">Radeau and Bertelson, 1987</xref>; <xref rid="B39" ref-type="bibr">Recanzone, 2003</xref>), or top down cognitive factors such as semantic congruency (<xref rid="B27" ref-type="bibr">Laurienti et al., 2004</xref>). For example, incongruent audio–visual spatial information shows a localisation bias toward visual information, in the ventriloquist effect, even when cued to the auditory stimulus (<xref rid="B8" ref-type="bibr">Bermant and Welch, 1976</xref>; <xref rid="B10" ref-type="bibr">Bertelson and Radeau, 1981</xref>) while separation of asynchronous audio–visual stimuli was perceived as shorter if presented in congruent rather than incongruent spatial locations (<xref rid="B45" ref-type="bibr">Soto-Faraco et al., 2002</xref>; <xref rid="B51" ref-type="bibr">Vroomen and de Gelder, 2003</xref>) with the auditory information appearing to dominate (<xref rid="B19" ref-type="bibr">Fendrich and Corballis, 2001</xref>; <xref rid="B46" ref-type="bibr">Soto-Faraco et al., 2004</xref>).</p><p>Considering this we manipulated the first task by providing either congruent multisensory stimuli, in which the sonification and visual presentation were associated (e.g., two-tone sonification and two visual lines) or incongruent (e.g., two-tone sonification and one visual line) to the listener. The task requirements were as before with the listener instructed to indicate how many visual lines had been sonified to create the stimulus. Based on the multisensory literature, we hypothesized that congruent audio–visual stimuli would facilitate superior performance in contrast to performance with both incongruent audio–visual and audio only stimuli.</p></sec><sec sec-type="materials|methods" id="s1"><title>Materials and Methods</title><sec><title>Participants</title><p>We recruited 36 participants (28 female) via an Undergraduate Research Assistant module. Participant age ranged from 18 to 25 years old (<italic>M</italic> = 20.17, <italic>SD</italic> = 1.30). All participants provided informed written consent, and had normal or corrected eyesight, normal hearing and educated to undergraduate level. Four participants self-reported as left handed and all were naïve to the principles of sonification. 12 participants didn’t return for the second part of the study and this is reflected in the analysis. The study was approved by the University of Bath Psychology Ethics Committee (#13-204).</p></sec><sec><title>Materials and Stimulus Design</title><p>Visual stimuli were created in Adobe Photoshop 3.0 with the sonifications using the principles of The vOICe (<xref rid="B31" ref-type="bibr">Meijer, 1992</xref>) algorithm. Frequency analysis of the sonifications was conducted in Cool Edit Pro 2.0 with all visual stimuli and sonifications presented in E-Prime 2.0 running on a Windows 7 PC. Sonifications were transmitted to the listener via Sennheiser HD 585 headphones. All statistical analysis was conducted using SPSS version 21.0.</p></sec><sec><title>Stimulus Design</title><p>In Photoshop a grid of 48 pixel × 1.5 pixel rows was overlaid on a black background. Solid white lines were drawn over the full x-axis of the background with width and interval dependent on the stimulus type. Example of each type of line can be seen in <bold>Figure <xref ref-type="fig" rid="F1">1</xref></bold>. For the parallel line stimuli two one-row lines, separated by the designated interval were created. The interval was varied from a two-row interval to a 42 row interval, with each interval gap increasing by two rows. The initial starting point was the center of the y-axis with each interval involving moving the top line up one row and the bottom line down 1 row from baseline or the previous stimulus. There were two types of single line stimuli. Filled stimuli took the parallel line stimuli and filled the gap between the two lines with white pixels. Thus the top and bottom lines were the same as the parallel line counterparts but with no interval between. The single line stimuli consisted of a line 2 rows thick (giving the same amount of white pixels as the parallel line). In total there were 23 parallel line, 24 single, and 24 filled stimulus images (two lines together at the central point of the y-axis was classified as a single line).</p><fig id="F1" position="float"><label>FIGURE 1</label><caption><p><bold>Illustration of the types of visual stimuli used for sonification.</bold> Two examples shown of parallel lines with different intervals, filled lines with different bandwidths, and single lines at different frequencies. Duration and frequency range of the sonifications also shown.</p></caption><graphic xlink:href="fpsyg-06-01522-g001"></graphic></fig><p>The lines were sonified using the following principles: the duration of each sonification, represented on the x-axis, was consistent for all stimuli (1000 ms), pitch was mapped to the y-axis with a range of 500 Hz (bottom) to 5000 Hz (top).White pixels were sonified at maximum volume (-65 dB) with black pixels silent. Each sonification therefore comprised of two complex tones at varying frequencies playing concurrently for 1000 ms (parallel lines), or one complex tone with the same top and bottom frequencies as the parallel line counterpart playing for 1000 ms (filled lines), or one complex tone at a consistent ‘visual’ width playing for 1000 ms (single line). Parallel line sonifications were categorized as consonant or dissonant based on the frequency range of the interval between the two lines.</p></sec><sec><title>Procedure</title><p>Participants watched a PowerPoint presentation with audio–visual examples of the sonification process with a brief introduction to its applications. Example parallel lines, plus the two types of single lines with their sonifications were included as well as an example of the task procedure. For each trial of the main task the listener was presented with a soundscape which had been sonified from either 1 or 2 visual lines. Their task was to indicate on the PC keyboard whether the sonification was of 1 or 2 lines. Participants were explicitly told in both the instructions and PowerPoint that a filled line was classed as a single line. There was no visual information or post-trial feedback given. Each experimental block consisted of 96 trials (48 (2 × 24) × parallel, 24 × filled, 24 × single) with trial order fully randomized within block and no repeated trials. There were four blocks in total, randomized across participants, to give 386 trials in total.</p><p>The audio–visual task had the same listener requirements as the audio-only task, that is, to indicate how many lines were used to create the sonification. For each trial the listener heard a soundscape sonified from one or two lines. At the same time an image of one or two white lines appeared on the PC monitor. The audio–visual presentation could either be congruent, where the number of lines matched over both modalities, or incongruent where there was a mismatch. The participants were informed that while it was a requisite to look at the screen for timing purposes they were not required to indicate how many visual lines they perceived, just the number of ‘lines’ in the soundscape. As with the audio-only task there was no feedback. Again there were 4 blocks of 96 randomized trials. Examples of the example trials in both conditions are shown in <bold>Figure <xref ref-type="fig" rid="F2">2</xref></bold>.</p><fig id="F2" position="float"><label>FIGURE 2</label><caption><p><bold>Illustration of four different trial types.</bold> What the participant sees on screen is shown in the top row. Spectrographs of the audio signal the participant hears is shown in the second row with the correct response in the third row. The two trials on the left are audio-only trials with the two on the right congruent and incongruent audio–visual trials.</p></caption><graphic xlink:href="fpsyg-06-01522-g002"></graphic></fig></sec></sec><sec><title>Results</title><p>Consider accuracy for the parallel line condition first. <bold>Figure <xref ref-type="fig" rid="F3">3</xref></bold> displays accuracy for individual parallel line frequencies, and clearly illustrates that the size of the interval between lines affects accurate recognition [<italic>F</italic>(8.52,298.04) = 21.937, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M1"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.385]. It is also clear that this cannot be solely due to proximity as some proximal lines (e.g., 498 Hz) are discriminated better than more distal lines (e.g., 3111 Hz), indicating that the predicted harmonic grouping is the relevant factor. <bold>Figure <xref ref-type="fig" rid="F3">3</xref></bold> also displays the pattern for consonant (<50%) and dissonant (>50%) stimuli which matches the predictions from the categorization based on consonance and dissonance. Analysis of variance on these seven groups, as shown in <bold>Figure <xref ref-type="fig" rid="F4">4</xref></bold>, again showed a main omnibus effect [<italic>F</italic>(3.19,111.52) = 42.182, <italic>p</italic> < 0.0001, <inline-formula><mml:math id="M2"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.547].</p><fig id="F3" position="float"><label>FIGURE 3</label><caption><p><bold>Correct response (%) for parallel line stimuli for each frequency gap prior to categorization into consonant and dissonant groups.</bold> Error bars show ±1 SEM.</p></caption><graphic xlink:href="fpsyg-06-01522-g003"></graphic></fig><fig id="F4" position="float"><label>FIGURE 4</label><caption><p><bold>Correct response (%) for parallel line discrimination with after categorization into consonant (blue) and dissonant (red) groups.</bold> Frequency ranges for each interval are shown on the x-axis. Error bars show ±1 SEM.</p></caption><graphic xlink:href="fpsyg-06-01522-g004"></graphic></fig><p>With harmonicity appearing the main factor in parallel line discrimination all relevant conditions were analyzed together: audio-only consonant, audio-only dissonant, audio–visual consonant congruent, audio–visual consonant incongruent, audio–visual dissonant congruent, and audio–visual dissonant incongruent. Results are shown in <bold>Figure <xref ref-type="fig" rid="F5">5</xref></bold> and <bold>Table <xref ref-type="table" rid="T1">1</xref></bold>. With accuracy as the D.V., an ANOVA, Greenhouse-Geisser corrected for violation of sphericity (𝜀 = 0.588), showed an omnibus main effect [<italic>F</italic>(2.94,64.69) = 19.162, <italic>p</italic> < 0.000, <inline-formula><mml:math id="M3"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.466] again displaying that, when factoring in audio–visual conditions, the size of the interval between parallel lines is influential in line discrimination. To assess where these differences lay planned contrasts, Bonferroni corrected for multiple comparisons, were conducted.</p><fig id="F5" position="float"><label>FIGURE 5</label><caption><p><bold>Correct response (%) for parallel line discrimination when trials used congruent and incongruent audio–visual stimuli.</bold> Means are shown for consonant (blue) and dissonant (red) intervals for audio-only and audio–visual conditions. Error bars show ±1 SEM.</p></caption><graphic xlink:href="fpsyg-06-01522-g005"></graphic></fig><table-wrap id="T1" position="float"><label>Table 1</label><caption><p>Correct response (%), for parallel line discrimination for consonant and dissonant stimuli in; audio-only, congruent audio–visual, and incongruent audio–visual conditions.</p></caption><table frame="hsides" rules="groups" cellspacing="5" cellpadding="5"><thead><tr><td valign="top" align="left" rowspan="1" colspan="1"></td><th valign="top" align="center" colspan="2" rowspan="1">Consonant<hr></hr></th><th valign="top" align="center" colspan="2" rowspan="1">Dissonant<hr></hr></th></tr><tr><td valign="top" align="left" rowspan="1" colspan="1"></td><th valign="top" align="left" rowspan="1" colspan="1">Mean</th><th valign="top" align="left" rowspan="1" colspan="1"><italic>SD</italic></th><th valign="top" align="left" rowspan="1" colspan="1">Mean</th><th valign="top" align="left" rowspan="1" colspan="1"><italic>SD</italic></th></tr></thead><tbody><tr><td valign="top" align="left" rowspan="1" colspan="1">Audio-only</td><td valign="top" align="left" rowspan="1" colspan="1">30.73</td><td valign="top" align="left" rowspan="1" colspan="1">22.86</td><td valign="top" align="left" rowspan="1" colspan="1">59.48</td><td valign="top" align="left" rowspan="1" colspan="1">19.54</td></tr><tr><td valign="top" align="left" rowspan="1" colspan="1"><bold>Audio–visual</bold></td><td valign="top" align="left" rowspan="1" colspan="1"></td><td valign="top" align="left" rowspan="1" colspan="1"></td><td valign="top" align="left" rowspan="1" colspan="1"></td><td valign="top" align="left" rowspan="1" colspan="1"></td></tr><tr><td valign="top" align="left" rowspan="1" colspan="1">Congruent</td><td valign="top" align="left" rowspan="1" colspan="1">42.75</td><td valign="top" align="left" rowspan="1" colspan="1">26.33</td><td valign="top" align="left" rowspan="1" colspan="1">70.58</td><td valign="top" align="left" rowspan="1" colspan="1">19.33</td></tr><tr><td valign="top" align="left" rowspan="1" colspan="1">Incongruent</td><td valign="top" align="left" rowspan="1" colspan="1">32.79</td><td valign="top" align="left" rowspan="1" colspan="1">25.85</td><td valign="top" align="left" rowspan="1" colspan="1">54.55</td><td valign="top" align="left" rowspan="1" colspan="1">24.16</td></tr><tr><td valign="top" align="left" rowspan="1" colspan="1"></td></tr></tbody></table></table-wrap><p>For trials where the stimuli were audio-only harmonicity had a large impact. Dissonant stimuli (<italic>M</italic> = 59.48), where the interval should not elicit any tonal confusion, were discriminated more successfully than consonant stimuli (<italic>M</italic> = 30.73) where harmonic relations should impact on performance [<italic>MD</italic> = 27.525, 95% CI(15.84,39.21), <italic>p</italic> < 0.0005]. The latter were also significantly below what would be expected by chance [<italic>t</italic>(35) = -5.058, <italic>p</italic> < 0.0005, <italic>d</italic> = 1.34] illustrating the magnitude of the ‘confusion’ caused by these harmonic relations.</p><p>Could this effect be anyway negated by using multisensory stimuli providing additional visual information? With the literature implying that multisensory binding requires some form of synchronicity we would only expect improved performance for audio–visual trials that were congruent, that is, provide the same line information via different modalities. The contrasts for the consonant stimuli showed no evidence of increased performance due to either congruent (<italic>M</italic> = 42.75) or incongruent (<italic>M</italic> = 32.79) audio–visual stimuli with significance levels of <italic>p</italic> = 0.797 and <italic>p</italic> = 0.984, respectively.</p><p>For dissonant stimuli, where performance in the audio-only condition was already significantly above chance [<italic>t</italic>(35) = 2.912, <italic>p</italic> = 0.006, <italic>d</italic> = 3.04] with no issues of harmonic relations we would expect an improvement in performance congruent trials in the audio–visual conditions. While the contrasts showed higher mean accuracy for the congruent condition (<italic>M</italic> = 70.58) and a lower one for the incongruent (<italic>M</italic> = 54.55), compared to the audio-only (<italic>M</italic> = 59.95) neither differences were significant with <italic>p</italic>-values of 0.445 and 0.984, respectively.</p><p>Secondly we considered whether proximity was an influence on discrimination of parallel lines, that is, would sonified lines closer together be less likely to be segregated into separate objects? Looking at the seven groups categorized by the frequency ranges shown in <bold>Figure <xref ref-type="fig" rid="F4">4</xref></bold>, we only contrasted within groups, that is, consonant versus consonant and dissonant versus dissonant. With the harmonicity effect having such a profound effect on performance comparisons between consonant and dissonant groups would naturally show a significant effect with the variance explained by these harmonic relations.</p><p>With accuracy as the dependent variable an ANOVA factoring in all consonant groups (audio-only, audio–visual congruent and audio–visual incongruent) showed an omnibus main effect for proximity [<italic>F</italic>(8,176) = 3.528, <italic>p</italic> = 0.001, <inline-formula><mml:math id="M4"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.138] with a separate ANOVA for dissonant groups showing similar [<italic>F</italic>(11,242) = 5.335, <italic>p</italic> = 0.001, <inline-formula><mml:math id="M5"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.195]. The Bonferroni corrected planned contrasts for both analyses tell a similar story. The only significant planned contrasts were between the congruent and incongruent audio–visual categories. For example, for consonant trials disregarding harmonicity, discrimination in the largest congruent category was better than for the two smallest incongruent categories (<italic>p</italic> = 0.008) and (<italic>p</italic> = 0.013), respectively. Dissonant trials in the smallest congruent group were better than for the smallest (<italic>p</italic> = 0.002) and second smallest (<italic>p</italic> = 0.018) incongruent groups. The second largest congruent elicited better scores than all four incongruent groups (smallest-to-largest, <italic>p</italic> = 0.026, <italic>p</italic> = 0.001, <italic>p</italic> = 0.008, <italic>p</italic> = 0.001), with the largest congruent group better than the smallest (<italic>p</italic> = 0.009) and largest (<italic>p</italic> = 0.009) incongruent. There were no significant contrasts within groups or involving the audio-only trials.</p><p>Analysis of the filled line data corroborates the lack of any effect of proximity. These lines retained the same top and bottom frequencies as the parallel lines but with the interval filled with white pixels/sonified noise. Without the intervals there can be no effect of harmonicity and therefore any differences are due to proximity or signal bandwidth. With all groups (7 × audio-only, 7 × audio–visual congruent, 7 × audio–visual incongruent) entered into an ANOVA there was a significant omnibus main effect [<italic>F</italic>(20,360) = 3.401, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M6"><mml:msubsup><mml:mi mathvariant="normal" mathcolor="black">η</mml:mi><mml:mi mathvariant="normal" mathcolor="black">p</mml:mi><mml:mn mathvariant="normal" mathcolor="black">2</mml:mn></mml:msubsup></mml:math></inline-formula> = 0.159]. However, while there were 17 significant contrasts at an alpha of <0.05 these were all between audio–visual congruent (good) and incongruent groups (poor) with no differences within groups or involving the audio-only condition.</p><p>In summary. When presented with audio-only stimuli where the interval had no harmonic relations the task was relatively easy with participants scoring above chance. However, when the interval does have harmonic relations, signified by tonal-fusion, the negative impact of this makes the task difficult with participants below chance levels. The use of audio–visual stimuli has little impact on lessening the effect of harmonicity and even when this effect is discounted, i.e., dissonant stimuli only, the congruent trials show a trend of better discrimination, but not reaching significance. Secondly, there is little evidence that proximity influences the discrimination of the sonifications with the only effects in this analysis being down to the use of congruent and incongruent audio–visual stimuli.</p></sec><sec><title>Discussion</title><p>In this study we evaluated whether feature segregation of sonified horizontal lines would be influenced by rules of ASA. Unlike simple stimuli used in auditory research, the sonifications here were complex, with wider interval marker bandwidths dictated by the visual features of the stimulus interacting with the principles of the sonification algorithm. However, even with this coarse representation, sonifications with consonant intervals demonstrated poor segregation as predicted by ASA. Secondly we assessed whether the provision of additional multisensory information would negate the effects of harmonicity. While congruent audio–visual information displayed a trend for superior feature segregation, relative to incongruent audio–visual and audio-only, this only reached significance for the former contrast.</p><p>The results fall broadly in line with what is predicted in the auditory literature (<xref rid="B35" ref-type="bibr">Plomp and Levelt, 1965</xref>; <xref rid="B12" ref-type="bibr">Bregman et al., 1990</xref>; <xref rid="B11" ref-type="bibr">Bregman, 1994</xref>) demonstrating the negative impact of consonance on feature segregation. Even when visual lines were almost the full height (y-axis) of the workspace apart, with associated sonifications separated by >3100 hz, harmonic relations elicited the perception of one object. While these findings are not too surprising they do emphasize the robustness of the effect to interval markers of varying complexity. The logarithmic frequency conversion of the algorithm renders visual lines of equal width as sonifications whose bandwidths are dependent on their elevation in the visual field. For example, in our study the frequency bandwidth of a two-pixel wide line at the top of the screen was over 800 hz greater than the equivalent line at the bottom of the screen. Within the somewhat sparse simultaneous two-tone discrimination literature in the auditory domain, in which visual factors are not applicable, this interval marker bandwidth variability is not assessed as stimuli parameters can be more controlled. Of course it would be interesting to evaluate how much variance between the two markers, in bandwidth and other features, would be required to reduce the consonance effect. There is certainly evidence that two-tone complexes are more easily resolved if the amplitude of one of the tones is more intense (<xref rid="B3" ref-type="bibr">Arehart and Rosengard, 1999</xref>) and this could have been evaluated in the present experiment by manipulating the shading of one of the visual lines.</p><p>Using The vOICe algorithm for the visual-to-auditory conversion necessitates a signal that is not static in the stereo field over time, that is, the signal initiates in the left headphone and pans across the stereo field to the right headphone over the duration of the scan. In a simultaneous two-tone pitch discrimination task <xref rid="B50" ref-type="bibr">Thurlow and Bernstein (1957)</xref> compared conditions where either the two tones were presented to the same ear (analogous to the present study), or presented to separate ears. Results showed little difference in discrimination for the five tested frequency levels when led to separate ears, however, when led to the same ear equivalent performance was only for stimuli where masking effects were minimized. If The vOICe signal was led to separate ears with the low frequency line scanning right-to-left and the high frequency line left-to-right, would this negate the masking effects demonstrated in the study? It is certainly a consideration for future research.</p><p>Simultaneous two-tone discrimination has been evaluated in different users to assess individual and group differences. An obvious group to test is trained musicians as successful pitch discrimination is an essential tool in their skillset. <xref rid="B26" ref-type="bibr">Kleczkowski and Pluta (2012)</xref> demonstrated that trained musicians were able to discriminate pitches at narrower levels than non-musicians, with similar results for musicians resolving harmonics in inharmonic complexes (<xref rid="B34" ref-type="bibr">Plomp, 1976</xref>). Musicians have also shown higher levels of performance using sensory substitution devices with <xref rid="B23" ref-type="bibr">Haigh et al. (2013)</xref> reporting musical ability correlating with higher acuity in a task using the vOICe and the Snellen Tumbling ‘E’. All participants in the study were sighted and naïve to sensory substitution and yet demonstrated acuity approaching the legal blindness definition of 20/200. In a similar acuity test with blind participants trained to use the device even lower acuity was reported (<xref rid="B49" ref-type="bibr">Striem-Amit et al., 2012</xref>) illustrating not only the effect of training but also potentialities due to superior auditory abilities, such as frequency discrimination (<xref rid="B41" ref-type="bibr">Roder et al., 1999</xref>; <xref rid="B52" ref-type="bibr">Wan et al., 2010</xref>), posited to be found in these populations. It would therefore be of great interest to test whether highly trained blind users of The vOICe could overcome the effect of consonance found in the present study. If so, this psychophysical test will provide solid evidence whether, through perceptual learning, the user is truly ‘seeing’ the sound or just hearing it. Considering the strength of consonance reported, it is highly doubtful that the effect would be negated in auditory domain and thus any difference in performance in these populations would imply a percept beyond audition.</p><p>The strength of the consonance effect is further exemplified by the limited influence of congruent and incongruent visual information. In speeded classification tasks evaluating cross-modal congruency, classification of visual stimuli as ‘high’ or ‘low’ has been shown to be more rapid if accompanied by tones that were congruent rather than incongruent (<xref rid="B9" ref-type="bibr">Bernstein and Edelstein, 1971</xref>; <xref rid="B6" ref-type="bibr">Ben-Artzi and Marks, 1995</xref>) with <xref rid="B18" ref-type="bibr">Evans and Treisman (2010)</xref> showing that cross-modal mappings between audio and visual stimuli are automatic and affect performance even when irrelevant to the task. This integration of temporally synchronous multisensory information is weighted to specific modalities as a function of the task (<xref rid="B47" ref-type="bibr">Spence, 2011</xref>), drawing support from a metamodal theory of the brain organization (<xref rid="B32" ref-type="bibr">Pascual-Leone and Hamilton, 2001</xref>). Here the brain is viewed as a task based machine with brain areas that are functionally optimal for particular computations; auditory areas for temporal tasks and visual for spatial (<xref rid="B36" ref-type="bibr">Proulx et al., 2014</xref>). In the present study the discrimination task can be considered spatial as the temporal features of the stimuli were identical. True to the metamodal theory, this adds weight to the visual information. If the audio–visual stimuli were congruent this should elicit better performance, and while the data showed a trend for this, it was not strong enough to bring discrimination of consonant stimuli above chance levels. Conversely, the incongruent visual information should reduce performance as there is extra weight attributed to the irrelevant distractor but again this trend was non-significant. Naturally with no access to visual information the blind users would not experience this audio–visual congruence, however, this could be tested using congruent and incongruent tactile stimuli. Simple methods such as embossed print outs of the visual workspace, or more technological based techniques involving haptic displays could be utilized to give multisensory information.</p><p>The results of our experiment show that the influence of consonance on object segregation is applicable to the sonification of coarse visual objects, but how can this information be suitably utilized? One approach to sonify a visual computer workspace is to evaluate the original visual stimulus and a spectrograph of it. Comparing these to the auditory representation would allow an evaluation of any potential auditory masking that might arise. This could include the direct mapping of spectrographs over the visual workspace in the development stage. Secondly, it would be interesting to evaluate how much consonance impacts on the use of sensory substitution devices when used in real-time. In such scenarios the sonified visual field updates at the device scan rate (1000 ms at default) to provide a continuous stream of ‘static’ frames. Thus, two parallel line sonifications masked in the first frame would only remain masked in the following frame if the device sensor, and background, remained static. For example, if the sensor was closer to the object in the second frame the parallel lines would be more disparate on the y-axis, the auditory interval increased, and the consonance negated.</p><p>A second consideration is variability and density of information provided in real-time device use. The present study utilized relatively simple stimuli, equal in all properties aside from auditory frequency, on a silent background. Objects encountered in everyday use are likely to be considerably more complex and therefore, even with masking, there should be sufficient unmasked signal to facilitate recognition. Indeed in a simple object recognition task using The vOICe, <xref rid="B14" ref-type="bibr">Brown et al. (2014)</xref> demonstrated equitable performance for degraded signals with limited information in contrast to more detailed stimuli.</p><p>Considering the above it seems unlikely that the negative effects of consonance would impact on real-time use of sensory substitution devices, although it should be considered if using static objects in early training paradigms. Interestingly, however, reducing dissonance has already been applied to visual-to-auditory sensory substitution. The EyeMusic uses similar conversion principles to the vOICe as well as coding basic colors to musical instruments (<xref rid="B1" ref-type="bibr">Abboud et al., 2014</xref>). In an attempt to make device use less uncomfortable, a pentatonic scale, alongside a reduced frequency range, is used to reduce dissonance. This is logical considering dissonance in audition is associated with a harsh perceptual experience. However, as we have demonstrated in our simple object discrimination task, dissonance appears important in feature segregation and it may be worth evaluating if there would be a comfort-function trade off in such tasks using EyeMusic.</p></sec><sec><title>Conflict of Interest Statement</title><p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p></sec></body><back><ack><p>This work was supported in part by a grant from the EPSRC to MP (EP/J017205/1) and the EPSRC Doctoral Training Account studentship at Queen Mary University of London to AS.</p></ack><ref-list><title>References</title><ref id="B1"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Abboud</surname><given-names>S.</given-names></name><name><surname>Hanassy</surname><given-names>S.</given-names></name><name><surname>Levy-Tzedek</surname><given-names>S.</given-names></name><name><surname>Maidenbaum</surname><given-names>S.</given-names></name><name><surname>Amedi</surname><given-names>A.</given-names></name></person-group> (<year>2014</year>). <article-title>EyeMusic: Introducing a “visual” colorful experience for the blind using auditory sensory substitution.</article-title><source><italic>Restor. Neurol. Neurosci.</italic></source><volume>32</volume><fpage>247</fpage>–<lpage>257</lpage>. <pub-id pub-id-type="doi">10.3233/RNN-130338</pub-id><pub-id pub-id-type="pmid">24398719</pub-id></mixed-citation></ref><ref id="B2"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Amedi</surname><given-names>A.</given-names></name><name><surname>Stern</surname><given-names>W. M.</given-names></name><name><surname>Camprodon</surname><given-names>J. A.</given-names></name><name><surname>Bermpohl</surname><given-names>F.</given-names></name><name><surname>Merabet</surname><given-names>L.</given-names></name><name><surname>Rotman</surname><given-names>S.</given-names></name><etal></etal></person-group> (<year>2007</year>). <article-title>Shape conveyed by visual-to-auditory sensory substitution activates the lateral occipital complex.</article-title><source><italic>Nat. Neurosci.</italic></source><volume>10</volume><fpage>687</fpage>–<lpage>689</lpage>. <pub-id pub-id-type="doi">10.1038/nn1912</pub-id><pub-id pub-id-type="pmid">17515898</pub-id></mixed-citation></ref><ref id="B3"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Arehart</surname><given-names>K. H.</given-names></name><name><surname>Rosengard</surname><given-names>P. F.</given-names></name></person-group> (<year>1999</year>). <article-title>Psychometric functions for discrimination of two-component complex tones in listeners with normal hearing and listeners with hearing loss.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>105</volume><fpage>2454</fpage>–<lpage>2465</lpage>. <pub-id pub-id-type="doi">10.1121/1.426850</pub-id><pub-id pub-id-type="pmid">10212426</pub-id></mixed-citation></ref><ref id="B4"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Auvray</surname><given-names>M.</given-names></name><name><surname>Hanneton</surname><given-names>S.</given-names></name><name><surname>O’Regan</surname><given-names>J. K.</given-names></name></person-group> (<year>2007</year>). <article-title>Learning to perceive with a visuo-auditory substitution system: localisation and object recognition with ‘the vOICe.’</article-title><source><italic>Perception</italic></source><volume>36</volume><fpage>416</fpage>–<lpage>430</lpage>. <pub-id pub-id-type="doi">10.1068/p5631</pub-id><pub-id pub-id-type="pmid">17455756</pub-id></mixed-citation></ref><ref id="B5"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bach-y-Rita</surname><given-names>P.</given-names></name><name><surname>Kercel</surname><given-names>S. W.</given-names></name></person-group> (<year>2003</year>). <article-title>Sensory substitution and the human–machine interface.</article-title><source><italic>Trends Cogn. Sci.</italic></source><volume>7</volume><fpage>541</fpage>–<lpage>546</lpage>. <pub-id pub-id-type="doi">10.1016/j.tics.2003.10.013</pub-id><pub-id pub-id-type="pmid">14643370</pub-id></mixed-citation></ref><ref id="B6"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ben-Artzi</surname><given-names>E.</given-names></name><name><surname>Marks</surname><given-names>L. E.</given-names></name></person-group> (<year>1995</year>). <article-title>Visual-auditory interaction in speeded classification: role of stimulus difference.</article-title><source><italic>Percept. Psychophys.</italic></source><volume>57</volume><fpage>1151</fpage>–<lpage>1162</lpage>. <pub-id pub-id-type="doi">10.3758/BF03208371</pub-id><pub-id pub-id-type="pmid">8539090</pub-id></mixed-citation></ref><ref id="B7"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ben-Av</surname><given-names>M. B.</given-names></name><name><surname>Sagi</surname><given-names>D.</given-names></name><name><surname>Braun</surname><given-names>J.</given-names></name></person-group> (<year>1992</year>). <article-title>Visual attention and perceptual grouping.</article-title><source><italic>Percept. Psychophys.</italic></source><volume>52</volume><fpage>277</fpage>–<lpage>294</lpage>. <pub-id pub-id-type="doi">10.3758/BF03209145</pub-id><pub-id pub-id-type="pmid">1408639</pub-id></mixed-citation></ref><ref id="B8"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bermant</surname><given-names>R. I.</given-names></name><name><surname>Welch</surname><given-names>R. B.</given-names></name></person-group> (<year>1976</year>). <article-title>Effect of degree of separation of visual-auditory stimulus and eye position upon spatial interaction of vision and audition.</article-title><source><italic>Percept. Mot. Skills</italic></source><volume>42</volume><fpage>487</fpage>–<lpage>493</lpage>. <pub-id pub-id-type="doi">10.2466/pms.1976.42.2.487</pub-id><pub-id pub-id-type="pmid">980644</pub-id></mixed-citation></ref><ref id="B9"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bernstein</surname><given-names>I. H.</given-names></name><name><surname>Edelstein</surname><given-names>B. A.</given-names></name></person-group> (<year>1971</year>). <article-title>Effects of some variations in auditory input upon visual choice reaction time.</article-title><source><italic>J. Exp. Psychol.</italic></source><volume>87</volume><fpage>241</fpage>–<lpage>247</lpage>. <pub-id pub-id-type="doi">10.1037/h0030524</pub-id><pub-id pub-id-type="pmid">5542226</pub-id></mixed-citation></ref><ref id="B10"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bertelson</surname><given-names>P.</given-names></name><name><surname>Radeau</surname><given-names>M.</given-names></name></person-group> (<year>1981</year>). <article-title>Cross-modal bias and perceptual fusion with auditory-visual spatial discordance.</article-title><source><italic>Percept. Psychophys.</italic></source><volume>29</volume><fpage>578</fpage>–<lpage>584</lpage>. <pub-id pub-id-type="doi">10.3758/BF03207374</pub-id><pub-id pub-id-type="pmid">7279586</pub-id></mixed-citation></ref><ref id="B11"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Bregman</surname><given-names>A. S.</given-names></name></person-group> (<year>1994</year>). <source><italic>Auditory Scene Analysis: The Perceptual Organization of Sound</italic>.</source><publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>MIT press</publisher-name>.</mixed-citation></ref><ref id="B12"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bregman</surname><given-names>A. S.</given-names></name><name><surname>Liao</surname><given-names>C.</given-names></name><name><surname>Levitan</surname><given-names>R.</given-names></name></person-group> (<year>1990</year>). <article-title>Auditory grouping based on fundamental frequency and formant peak frequency.</article-title><source><italic>Can. J. Psychol.</italic></source><volume>44</volume><fpage>400</fpage>–<lpage>413</lpage>. <pub-id pub-id-type="doi">10.1037/h0084255</pub-id><pub-id pub-id-type="pmid">2224643</pub-id></mixed-citation></ref><ref id="B13"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Brown</surname><given-names>D. J.</given-names></name><name><surname>Macpherson</surname><given-names>T.</given-names></name><name><surname>Ward</surname><given-names>J.</given-names></name></person-group> (<year>2011</year>). <article-title>Seeing with sound? Exploring different characteristics of a visual-to-auditory sensory substitution device.</article-title><source><italic>Perception</italic></source><volume>40</volume><fpage>1120</fpage>–<lpage>1135</lpage>. <pub-id pub-id-type="doi">10.1068/p6952</pub-id><pub-id pub-id-type="pmid">22208131</pub-id></mixed-citation></ref><ref id="B14"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Brown</surname><given-names>D. J.</given-names></name><name><surname>Simpson</surname><given-names>A. J. R.</given-names></name><name><surname>Proulx</surname><given-names>M. J.</given-names></name></person-group> (<year>2014</year>). <article-title>Visual objects in the auditory system in sensory substitution: how much information do we need?</article-title><source><italic>Multisens. Res.</italic></source><volume>27</volume><fpage>337</fpage>–<lpage>357</lpage>. <pub-id pub-id-type="doi">10.1163/22134808-00002462</pub-id><pub-id pub-id-type="pmid">25693300</pub-id></mixed-citation></ref><ref id="B15"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cherry</surname><given-names>C.</given-names></name></person-group> (<year>1953</year>). <article-title>Some experiments on the recognition of speech, with one and with two ears.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>25</volume><fpage>975</fpage>–<lpage>979</lpage>. <pub-id pub-id-type="doi">10.1121/1.1907229</pub-id></mixed-citation></ref><ref id="B16"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Collignon</surname><given-names>O.</given-names></name><name><surname>Lassonde</surname><given-names>M.</given-names></name><name><surname>Lepore</surname><given-names>F.</given-names></name><name><surname>Bastien</surname><given-names>D.</given-names></name><name><surname>Veraart</surname><given-names>C.</given-names></name></person-group> (<year>2007</year>). <article-title>Functional cerebral reorganization for auditory spatial processing and auditory substitution of vision in early blind subjects.</article-title><source><italic>Cereb. Cortex</italic></source><volume>17</volume><fpage>457</fpage>–<lpage>465</lpage>. <pub-id pub-id-type="doi">10.1093/cercor/bhj162</pub-id><pub-id pub-id-type="pmid">16581983</pub-id></mixed-citation></ref><ref id="B17"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Driver</surname><given-names>J.</given-names></name><name><surname>Baylis</surname><given-names>G. C.</given-names></name></person-group> (<year>1989</year>). <article-title>Movement and visual attention: the spotlight metaphor breaks down.</article-title><source><italic>J. Exp. Psychol. Hum. Percept. Perform.</italic></source><volume>15</volume><fpage>448</fpage>–<lpage>456</lpage>. <pub-id pub-id-type="doi">10.1037/h0090403</pub-id><pub-id pub-id-type="pmid">2527954</pub-id></mixed-citation></ref><ref id="B18"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Evans</surname><given-names>K. K.</given-names></name><name><surname>Treisman</surname><given-names>A.</given-names></name></person-group> (<year>2010</year>). <article-title>Natural cross-modal mappings between visual and auditory features.</article-title><source><italic>J. Vis.</italic></source><volume>10</volume><fpage>1</fpage>–<lpage>12</lpage>. <pub-id pub-id-type="doi">10.1167/10.1.6</pub-id><pub-id pub-id-type="pmid">20143899</pub-id></mixed-citation></ref><ref id="B19"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Fendrich</surname><given-names>R.</given-names></name><name><surname>Corballis</surname><given-names>P. M.</given-names></name></person-group> (<year>2001</year>). <article-title>The temporal cross-capture of audition and vision.</article-title><source><italic>Percept. Psychophys.</italic></source><volume>63</volume><fpage>719</fpage>–<lpage>725</lpage>. <pub-id pub-id-type="doi">10.3758/BF03194432</pub-id><pub-id pub-id-type="pmid">11436740</pub-id></mixed-citation></ref><ref id="B20"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Frasnelli</surname><given-names>J.</given-names></name><name><surname>Collignon</surname><given-names>O.</given-names></name><name><surname>Voss</surname><given-names>P.</given-names></name><name><surname>Lepore</surname><given-names>F.</given-names></name></person-group> (<year>2011</year>). “<article-title>Crossmodal plasticity in sensory loss</article-title>,” in <source><italic>Progress in Brain Research, Part 1 Enhancing Performance for Action and Perception: Multisensory Integration, Neuroplasticity and Neuroprosthetics</italic></source>, <role>eds</role><person-group person-group-type="editor"><name><surname>Green</surname><given-names>A. M.</given-names></name><name><surname>Chapman</surname><given-names>C. E.</given-names></name><name><surname>Kalaska</surname><given-names>J. F.</given-names></name><name><surname>Lepore</surname><given-names>F.</given-names></name></person-group> (<publisher-loc>Amsterdam</publisher-loc>: <publisher-name>Elsivier</publisher-name>), <volume>233</volume>.</mixed-citation></ref><ref id="B21"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Frassinetti</surname><given-names>F.</given-names></name><name><surname>Bolognini</surname><given-names>N.</given-names></name><name><surname>Ladavas</surname><given-names>E.</given-names></name></person-group> (<year>2002</year>). <article-title>Enhancement of visual perception by crossmodal visuo-auditory interaction.</article-title><source><italic>Exp. Brain Res.</italic></source><volume>147</volume><fpage>332</fpage>–<lpage>343</lpage>. <pub-id pub-id-type="doi">10.1007/s00221-002-1262-y</pub-id><pub-id pub-id-type="pmid">12428141</pub-id></mixed-citation></ref><ref id="B22"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Frauenberger</surname><given-names>C.</given-names></name><name><surname>Stockman</surname><given-names>T.</given-names></name></person-group> (<year>2009</year>). <article-title>Auditory display design—an investigation of a design pattern approach.</article-title><source><italic>Int. J. Hum. Comput. Stud.</italic></source><volume>67</volume><fpage>907</fpage>–<lpage>922</lpage>. <pub-id pub-id-type="doi">10.1016/j.ijhcs.2009.05.008</pub-id></mixed-citation></ref><ref id="B23"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Haigh</surname><given-names>A.</given-names></name><name><surname>Brown</surname><given-names>D. J.</given-names></name><name><surname>Meijer</surname><given-names>P.</given-names></name><name><surname>Proulx</surname><given-names>M. J.</given-names></name></person-group> (<year>2013</year>). <article-title>How well do you see what you hear? The acuity of visual-to-auditory sensory substitution.</article-title><source><italic>Front. Psychol.</italic></source><volume>4</volume>:<issue>330</issue><pub-id pub-id-type="doi">10.3389/fpsyg.2013.00330</pub-id></mixed-citation></ref><ref id="B24"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Iordanescu</surname><given-names>L.</given-names></name><name><surname>Guzman-Martinez</surname><given-names>E.</given-names></name><name><surname>Grabowecky</surname><given-names>M.</given-names></name><name><surname>Suzuki</surname><given-names>S.</given-names></name></person-group> (<year>2008</year>). <article-title>Characteristic sounds facilitate visual search.</article-title><source><italic>Psychon. Bull. Rev.</italic></source><volume>15</volume><fpage>548</fpage>–<lpage>554</lpage>. <pub-id pub-id-type="doi">10.3758/PBR.15.3.548</pub-id><pub-id pub-id-type="pmid">18567253</pub-id></mixed-citation></ref><ref id="B25"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Jiménez</surname><given-names>J.</given-names></name><name><surname>Olea</surname><given-names>J.</given-names></name><name><surname>Torres</surname><given-names>J.</given-names></name><name><surname>Alonso</surname><given-names>I.</given-names></name><name><surname>Harder</surname><given-names>D.</given-names></name><name><surname>Fischer</surname><given-names>K.</given-names></name></person-group> (<year>2009</year>). <article-title>Biography of louis braille and invention of the braille alphabet.</article-title><source><italic>Surv. Ophthalmol.</italic></source><volume>54</volume><fpage>142</fpage>–<lpage>149</lpage>. <pub-id pub-id-type="doi">10.1016/j.survophthal.2008.10.006</pub-id><pub-id pub-id-type="pmid">19171217</pub-id></mixed-citation></ref><ref id="B26"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kleczkowski</surname><given-names>P.</given-names></name><name><surname>Pluta</surname><given-names>M.</given-names></name></person-group> (<year>2012</year>). <article-title>Frequency discrimination in a simultaneous two-tone signal.</article-title><source><italic>Acta Phys. Pol. Ser. A Gen. Phys.</italic></source><volume>121</volume> A120.</mixed-citation></ref><ref id="B27"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Laurienti</surname><given-names>P. J.</given-names></name><name><surname>Kraft</surname><given-names>R. A.</given-names></name><name><surname>Maldjian</surname><given-names>J. A.</given-names></name><name><surname>Burdette</surname><given-names>J. H.</given-names></name><name><surname>Wallace</surname><given-names>M. T.</given-names></name></person-group> (<year>2004</year>). <article-title>Semantic congruence is a critical factor in multisensory behavioral performance.</article-title><source><italic>Exp. Brain Res.</italic></source><volume>158</volume><fpage>405</fpage>–<lpage>414</lpage>. <pub-id pub-id-type="doi">10.1007/s00221-004-1913-2</pub-id><pub-id pub-id-type="pmid">15221173</pub-id></mixed-citation></ref><ref id="B28"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>MacDonald</surname><given-names>D.</given-names></name><name><surname>Stockman</surname><given-names>T.</given-names></name></person-group> (<year>2014</year>). <source><italic>Towards a Method, Techniques and Tools to Support the Development of Auditory Displays. HCI Engineering Charting the Way towards Methods and Tools for Advanced Interactive Systems</italic></source>, <volume>29</volume> Available at: <ext-link ext-link-type="uri" xlink:href="http://hci-engineering.di.uminho.pt/Program_files/MacDonald.pdf">http://hci-engineering.di.uminho.pt/Program_files/MacDonald.pdf</ext-link></mixed-citation></ref><ref id="B29"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Maidenbaum</surname><given-names>S.</given-names></name><name><surname>Levy-Tzedek</surname><given-names>S.</given-names></name><name><surname>Chebat</surname><given-names>D. R.</given-names></name><name><surname>Amedi</surname><given-names>A.</given-names></name></person-group> (<year>2013</year>). <article-title>Increasing accessibility to the blind of virtual environments, using a virtual mobility aid based on the “EyeCane”: feasibility study.</article-title><source><italic>PLoS ONE</italic></source><volume>8</volume>:<issue>e72555</issue><pub-id pub-id-type="doi">10.1371/journal.pone.0072555</pub-id></mixed-citation></ref><ref id="B30"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Marks</surname><given-names>L. E.</given-names></name></person-group> (<year>1974</year>). <article-title>On associations of light and sound: the mediation of brightness, pitch, and loudness.</article-title><source><italic>Am. J. Psychol.</italic></source><volume>87</volume><fpage>173</fpage>–<lpage>188</lpage>. <pub-id pub-id-type="doi">10.2307/1422011</pub-id><pub-id pub-id-type="pmid">4451203</pub-id></mixed-citation></ref><ref id="B31"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Meijer</surname><given-names>P.</given-names></name></person-group> (<year>1992</year>). <article-title>An experimental system for auditory image representations.</article-title><source><italic>IEEE Trans. Biomed. Eng.</italic></source><volume>39</volume><fpage>112</fpage>–<lpage>121</lpage>. <pub-id pub-id-type="doi">10.1109/10.121642</pub-id><pub-id pub-id-type="pmid">1612614</pub-id></mixed-citation></ref><ref id="B32"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Pascual-Leone</surname><given-names>A.</given-names></name><name><surname>Hamilton</surname><given-names>R.</given-names></name></person-group> (<year>2001</year>). <article-title>The metamodal organization of the brain.</article-title><source><italic>Prog. Brain Res.</italic></source><volume>134</volume><fpage>427</fpage>–<lpage>445</lpage>. <pub-id pub-id-type="doi">10.1016/S0079-6123(01)34028-1</pub-id><pub-id pub-id-type="pmid">11702559</pub-id></mixed-citation></ref><ref id="B33"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Plomp</surname><given-names>R.</given-names></name></person-group> (<year>1967</year>). <article-title>Pitch of complex tones.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>41</volume><fpage>1526</fpage>–<lpage>1533</lpage>. <pub-id pub-id-type="doi">10.1121/1.1910515</pub-id><pub-id pub-id-type="pmid">6075560</pub-id></mixed-citation></ref><ref id="B34"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Plomp</surname><given-names>R.</given-names></name></person-group> (<year>1976</year>). <source><italic>Aspects of Tone Perception: A Psychophysical Study</italic>.</source><publisher-loc>New York</publisher-loc>: <publisher-name>Academic Press</publisher-name>.</mixed-citation></ref><ref id="B35"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Plomp</surname><given-names>R.</given-names></name><name><surname>Levelt</surname><given-names>W. J. M.</given-names></name></person-group> (<year>1965</year>). <article-title>Tonal consonance and critical bandwidth.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>38</volume><fpage>548</fpage>–<lpage>560</lpage>. <pub-id pub-id-type="doi">10.1121/1.1909741</pub-id><pub-id pub-id-type="pmid">5831012</pub-id></mixed-citation></ref><ref id="B36"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Proulx</surname><given-names>M. J.</given-names></name><name><surname>Brown</surname><given-names>D. J.</given-names></name><name><surname>Pasqualotto</surname><given-names>A.</given-names></name><name><surname>Meijer</surname><given-names>P.</given-names></name></person-group> (<year>2014</year>). <article-title>Multisensory perceptual learning and sensory substitution.</article-title><source><italic>Neurosci. Biobehav. Rev.</italic></source><volume>41</volume><fpage>16</fpage>–<lpage>25</lpage>. <pub-id pub-id-type="doi">10.1016/j.neubiorev.2012.11.017</pub-id><pub-id pub-id-type="pmid">23220697</pub-id></mixed-citation></ref><ref id="B37"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Proulx</surname><given-names>M. J.</given-names></name><name><surname>Stoerig</surname><given-names>P.</given-names></name><name><surname>Ludowig</surname><given-names>E.</given-names></name><name><surname>Knoll</surname><given-names>I.</given-names></name></person-group> (<year>2008</year>). <article-title>Seeing ‘Where’ through the Ears: effects of learning-by-doing and long-term sensory deprivation on localization based on image-to-sound substitution.</article-title><source><italic>PLoS ONE</italic></source><volume>3</volume>:<issue>e1840</issue><pub-id pub-id-type="doi">10.1371/journal.pone.0001840</pub-id></mixed-citation></ref><ref id="B38"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Radeau</surname><given-names>M.</given-names></name><name><surname>Bertelson</surname><given-names>P.</given-names></name></person-group> (<year>1987</year>). <article-title>Auditory-visual interaction and the timing of inputs. Thomas (1941) revisited.</article-title><source><italic>Psychol. Res.</italic></source><volume>49</volume><fpage>17</fpage>–<lpage>22</lpage>.<pub-id pub-id-type="pmid">3615744</pub-id></mixed-citation></ref><ref id="B39"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Recanzone</surname><given-names>G. H.</given-names></name></person-group> (<year>2003</year>). <article-title>Auditory influences on visual temporal rate perception.</article-title><source><italic>J. Neurophysiol.</italic></source><volume>89</volume><fpage>1078</fpage>–<lpage>1093</lpage>. <pub-id pub-id-type="doi">10.1152/jn.00706.2002</pub-id><pub-id pub-id-type="pmid">12574482</pub-id></mixed-citation></ref><ref id="B40"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Renier</surname><given-names>L.</given-names></name><name><surname>Collignon</surname><given-names>O.</given-names></name><name><surname>Poirier</surname><given-names>C.</given-names></name><name><surname>Tranduy</surname><given-names>D.</given-names></name><name><surname>Vanlierde</surname><given-names>A.</given-names></name><name><surname>Bol</surname><given-names>A.</given-names></name><etal></etal></person-group> (<year>2005</year>). <article-title>Cross-modal activation of visual cortex during depth perception using auditory substitution of vision.</article-title><source><italic>Neuroimage</italic></source><volume>26</volume><fpage>573</fpage>–<lpage>580</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuroimage.2005.01.047</pub-id><pub-id pub-id-type="pmid">15907314</pub-id></mixed-citation></ref><ref id="B41"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Roder</surname><given-names>B.</given-names></name><name><surname>Teder-Salejarvi</surname><given-names>W.</given-names></name><name><surname>Sterr</surname><given-names>A.</given-names></name><name><surname>Rosler</surname><given-names>F.</given-names></name><name><surname>Hillyard</surname><given-names>S. A.</given-names></name><name><surname>Neville</surname><given-names>H. J.</given-names></name></person-group> (<year>1999</year>). <article-title>Improved auditory spatial tuning in blind humans.</article-title><source><italic>Nature</italic></source><volume>400</volume><fpage>162</fpage>–<lpage>166</lpage>. <pub-id pub-id-type="doi">10.1038/22106</pub-id><pub-id pub-id-type="pmid">10408442</pub-id></mixed-citation></ref><ref id="B42"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rowell</surname><given-names>J.</given-names></name><name><surname>Ungar</surname><given-names>S.</given-names></name></person-group> (<year>2003</year>). <article-title>The world of touch: an international survey of tactile maps. Part 1: production.</article-title><source><italic>Br. J. Vis. Impair.</italic></source><volume>21</volume><fpage>98</fpage>–<lpage>104</lpage>.</mixed-citation></ref><ref id="B43"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Sharpee</surname><given-names>T. O.</given-names></name><name><surname>Atencio</surname><given-names>C. A.</given-names></name><name><surname>Schreiner</surname><given-names>C. E.</given-names></name></person-group> (<year>2011</year>). <article-title>Hierarchical representations in the auditory cortex.</article-title><source><italic>Curr. Opin. Neurobiol.</italic></source><volume>21</volume><fpage>761</fpage>–<lpage>767</lpage>. <pub-id pub-id-type="doi">10.1016/j.conb.2011.05.027</pub-id><pub-id pub-id-type="pmid">21704508</pub-id></mixed-citation></ref><ref id="B44"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Shower</surname><given-names>E.</given-names></name><name><surname>Biddulph</surname><given-names>R.</given-names></name></person-group> (<year>1931</year>). <article-title>Differential pitch sensitivity of the ear.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>3</volume><fpage>7</fpage>–<lpage>7</lpage>. <pub-id pub-id-type="doi">10.1121/1.1915561</pub-id></mixed-citation></ref><ref id="B45"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Soto-Faraco</surname><given-names>S.</given-names></name><name><surname>Lyons</surname><given-names>J.</given-names></name><name><surname>Gazzaniga</surname><given-names>M.</given-names></name><name><surname>Spence</surname><given-names>C.</given-names></name><name><surname>Kingstone</surname><given-names>A.</given-names></name></person-group> (<year>2002</year>). <article-title>The ventriloquist in motion: illusory capture of dynamic information across sensory modalities.</article-title><source><italic>Brain Res. Cogn. Brain Res.</italic></source><volume>14</volume><fpage>139</fpage>–<lpage>146</lpage>. <pub-id pub-id-type="doi">10.1016/S0926-6410(02)00068-X</pub-id><pub-id pub-id-type="pmid">12063137</pub-id></mixed-citation></ref><ref id="B46"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Soto-Faraco</surname><given-names>S.</given-names></name><name><surname>Spence</surname><given-names>C.</given-names></name><name><surname>Kingstone</surname><given-names>A.</given-names></name></person-group> (<year>2004</year>). <article-title>Cross-modal dynamic capture: congruency effects in the perception of motion across sensory modalities.</article-title><source><italic>J. Exp. Psychol. Hum. Percept. Perform.</italic></source><volume>30</volume><fpage>330</fpage>–<lpage>345</lpage>. <pub-id pub-id-type="doi">10.1037/0096-1523.30.2.330</pub-id><pub-id pub-id-type="pmid">15053692</pub-id></mixed-citation></ref><ref id="B47"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Spence</surname><given-names>C.</given-names></name></person-group> (<year>2011</year>). <article-title>Crossmodal correspondences: a tutorial review.</article-title><source><italic>Atten. Percept. Psychophys.</italic></source><volume>73</volume><fpage>971</fpage>–<lpage>995</lpage>. <pub-id pub-id-type="doi">10.3758/s13414-010-0073-7</pub-id><pub-id pub-id-type="pmid">21264748</pub-id></mixed-citation></ref><ref id="B48"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Striem-Amit</surname><given-names>E.</given-names></name><name><surname>Amedi</surname><given-names>A.</given-names></name></person-group> (<year>2014</year>). <article-title>Visual cortex extrastriate body-selective area activation in congenitally blind people “seeing” by using sounds.</article-title><source><italic>Curr. Biol.</italic></source><volume>24</volume><fpage>687</fpage>–<lpage>692</lpage>. <pub-id pub-id-type="doi">10.1016/j.cub.2014.02.010</pub-id><pub-id pub-id-type="pmid">24613309</pub-id></mixed-citation></ref><ref id="B49"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Striem-Amit</surname><given-names>E.</given-names></name><name><surname>Guendelman</surname><given-names>M.</given-names></name><name><surname>Amedi</surname><given-names>A.</given-names></name></person-group> (<year>2012</year>). <article-title>‘Visual’ acuity of the congenitally blind using visual-to-auditory sensory substitution.</article-title><source><italic>PLoS ONE</italic></source><volume>7</volume>:<issue>e33136</issue><pub-id pub-id-type="doi">10.1371/journal.pone.0033136</pub-id></mixed-citation></ref><ref id="B50"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Thurlow</surname><given-names>W. R.</given-names></name><name><surname>Bernstein</surname><given-names>S.</given-names></name></person-group> (<year>1957</year>). <article-title>Simultaneous two-tone pitch discrimination.</article-title><source><italic>J. Acoust. Soc. Am.</italic></source><volume>29</volume><fpage>515</fpage>–<lpage>519</lpage>. <pub-id pub-id-type="doi">10.1121/1.1908946</pub-id></mixed-citation></ref><ref id="B51"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vroomen</surname><given-names>J.</given-names></name><name><surname>de Gelder</surname><given-names>B.</given-names></name></person-group> (<year>2003</year>). <article-title>Visual motion influences the contingent auditory motion aftereffect.</article-title><source><italic>Psychol. Sci.</italic></source><volume>14</volume><fpage>357</fpage>–<lpage>361</lpage>. <pub-id pub-id-type="doi">10.1111/1467-9280.24431</pub-id><pub-id pub-id-type="pmid">12807410</pub-id></mixed-citation></ref><ref id="B52"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wan</surname><given-names>C. Y.</given-names></name><name><surname>Wood</surname><given-names>A. G.</given-names></name><name><surname>Reutens</surname><given-names>D. C.</given-names></name><name><surname>Wilson</surname><given-names>S. J.</given-names></name></person-group> (<year>2010</year>). <article-title>Early but not late-blindness leads to enhanced auditory perception.</article-title><source><italic>Neuropsychologia</italic></source><volume>48</volume><fpage>344</fpage>–<lpage>348</lpage>. <pub-id pub-id-type="doi">10.1016/j.neuropsychologia.2009.08.016</pub-id><pub-id pub-id-type="pmid">19703481</pub-id></mixed-citation></ref><ref id="B53"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ward</surname><given-names>J.</given-names></name><name><surname>Meijer</surname><given-names>P.</given-names></name></person-group> (<year>2010</year>). <article-title>Visual experiences in the blind induced by an auditory sensory substitution device.</article-title><source><italic>Conscious. Cogn.</italic></source><volume>19</volume><fpage>492</fpage>–<lpage>500</lpage>. <pub-id pub-id-type="doi">10.1016/j.concog.2009.10.006</pub-id><pub-id pub-id-type="pmid">19955003</pub-id></mixed-citation></ref><ref id="B54"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wever</surname><given-names>E.</given-names></name><name><surname>Wedell</surname><given-names>C.</given-names></name></person-group> (<year>1941</year>). <article-title>Pitch discrimination at high frequencies.</article-title><source><italic>Psychol. Bull.</italic></source><volume>38</volume><issue>727</issue>.</mixed-citation></ref></ref-list></back></pmc><affiliations><list><country><li>Royaume-Uni</li></country></list><tree><country name="Royaume-Uni"><noRegion><name sortKey="Brown, David J" sort="Brown, David J" uniqKey="Brown D" first="David J." last="Brown">David J. Brown</name></noRegion><name sortKey="Brown, David J" sort="Brown, David J" uniqKey="Brown D" first="David J." last="Brown">David J. Brown</name><name sortKey="Proulx, Michael J" sort="Proulx, Michael J" uniqKey="Proulx M" first="Michael J." last="Proulx">Michael J. Proulx</name><name sortKey="Simpson, Andrew J R" sort="Simpson, Andrew J R" uniqKey="Simpson A" first="Andrew J. R." last="Simpson">Andrew J. R. Simpson</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/HapticV1/Data/Pmc/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000731 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd -nk 000731 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= HapticV1
|flux= Pmc
|étape= Checkpoint
|type= RBID
|clé= PMC:4602098
|texte= Auditory scene analysis and sonified visual images. Does consonance negatively impact on object formation when using complex sonified stimuli?
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/RBID.i -Sk "pubmed:26528202" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd \
| NlmPubMed2Wicri -a HapticV1
| This area was generated with Dilib version V0.6.23. |  |