Links to Exploration step
Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">PowerPoint<sup>®</sup> Presentation Flaws and Failures: A Psychological Analysis</title><author><name sortKey="Kosslyn, Stephen M" sort="Kosslyn, Stephen M" uniqKey="Kosslyn S" first="Stephen M." last="Kosslyn">Stephen M. Kosslyn</name><affiliation><nlm:aff id="aff1"><institution>Center for Advanced Study in the Behavioral Sciences, Stanford University</institution><country>Stanford, CA USA</country></nlm:aff></affiliation></author><author><name sortKey="Kievit, Rogier A" sort="Kievit, Rogier A" uniqKey="Kievit R" first="Rogier A." last="Kievit">Rogier A. Kievit</name><affiliation><nlm:aff id="aff2"><institution>Department of Psychology, University of Amsterdam</institution><country>Amsterdam, Netherlands</country></nlm:aff></affiliation></author><author><name sortKey="Russell, Alexandra G" sort="Russell, Alexandra G" uniqKey="Russell A" first="Alexandra G." last="Russell">Alexandra G. Russell</name><affiliation><nlm:aff id="aff3"><institution>Department of Psychology, Stanford University</institution><country>Stanford, CA, USA</country></nlm:aff></affiliation></author><author><name sortKey="Shephard, Jennifer M" sort="Shephard, Jennifer M" uniqKey="Shephard J" first="Jennifer M." last="Shephard">Jennifer M. Shephard</name><affiliation><nlm:aff id="aff4"><institution>Division of Social Science, Harvard University</institution><country>Cambridge, MA, USA</country></nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">22822402</idno><idno type="pmc">3398435</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3398435</idno><idno type="RBID">PMC:3398435</idno><idno type="doi">10.3389/fpsyg.2012.00230</idno><date when="2012">2012</date><idno type="wicri:Area/Pmc/Corpus">000416</idno><idno type="wicri:explorRef" wicri:stream="Pmc" wicri:step="Corpus" wicri:corpus="PMC">000416</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">PowerPoint<sup>®</sup> Presentation Flaws and Failures: A Psychological Analysis</title><author><name sortKey="Kosslyn, Stephen M" sort="Kosslyn, Stephen M" uniqKey="Kosslyn S" first="Stephen M." last="Kosslyn">Stephen M. Kosslyn</name><affiliation><nlm:aff id="aff1"><institution>Center for Advanced Study in the Behavioral Sciences, Stanford University</institution><country>Stanford, CA USA</country></nlm:aff></affiliation></author><author><name sortKey="Kievit, Rogier A" sort="Kievit, Rogier A" uniqKey="Kievit R" first="Rogier A." last="Kievit">Rogier A. Kievit</name><affiliation><nlm:aff id="aff2"><institution>Department of Psychology, University of Amsterdam</institution><country>Amsterdam, Netherlands</country></nlm:aff></affiliation></author><author><name sortKey="Russell, Alexandra G" sort="Russell, Alexandra G" uniqKey="Russell A" first="Alexandra G." last="Russell">Alexandra G. Russell</name><affiliation><nlm:aff id="aff3"><institution>Department of Psychology, Stanford University</institution><country>Stanford, CA, USA</country></nlm:aff></affiliation></author><author><name sortKey="Shephard, Jennifer M" sort="Shephard, Jennifer M" uniqKey="Shephard J" first="Jennifer M." last="Shephard">Jennifer M. Shephard</name><affiliation><nlm:aff id="aff4"><institution>Division of Social Science, Harvard University</institution><country>Cambridge, MA, USA</country></nlm:aff></affiliation></author></analytic><series><title level="j">Frontiers in Psychology</title><idno type="eISSN">1664-1078</idno><imprint><date when="2012">2012</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>Electronic slideshow presentations are often faulted anecdotally, but little empirical work has documented their faults. In Study 1 we found that eight psychological principles are often violated in PowerPoint<sup>®</sup> slideshows, and are violated to similar extents across different fields – for example, academic research slideshows generally were no better or worse than business slideshows. In Study 2 we found that respondents reported having noticed, and having been annoyed by, specific problems in presentations arising from violations of particular psychological principles. Finally, in Study 3 we showed that observers are not highly accurate in recognizing when particular slides violated a specific psychological rule. Furthermore, even when they correctly identified the violation, they often could not explain the nature of the problem. In sum, the psychological foundations for effective slideshow presentation design are neither obvious nor necessarily intuitive, and presentation designers in all fields, from education to business to government, could benefit from explicit instruction in relevant aspects of psychology.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Allen, R C" uniqKey="Allen R">R. C. Allen</name></author><author><name sortKey="Rubin, M L" uniqKey="Rubin M">M. L. Rubin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Alvarez, G" uniqKey="Alvarez G">G. Alvarez</name></author><author><name sortKey="Franconeri, S" uniqKey="Franconeri S">S. Franconeri</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Apperson, J M" uniqKey="Apperson J">J. M. Apperson</name></author><author><name sortKey="Laws, E L" uniqKey="Laws E">E. L. Laws</name></author><author><name sortKey="Scepansky, J A" uniqKey="Scepansky J">J. A. Scepansky</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Apperson, J M" uniqKey="Apperson J">J. M. Apperson</name></author><author><name sortKey="Laws, E L" uniqKey="Laws E">E. L. Laws</name></author><author><name sortKey="Scepansky, J A" uniqKey="Scepansky J">J. A. Scepansky</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Aspillaga, M" uniqKey="Aspillaga M">M. Aspillaga</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Baddeley, A D" uniqKey="Baddeley A">A. D. Baddeley</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bartsch, R A" uniqKey="Bartsch R">R. A. Bartsch</name></author><author><name sortKey="Cobern, K M" uniqKey="Cobern K">K. M. Cobern</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Beets, S D" uniqKey="Beets S">S. D. Beets</name></author><author><name sortKey="Lobingier, P G" uniqKey="Lobingier P">P. G. Lobingier</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Buchel, C" uniqKey="Buchel C">C. Buchel</name></author><author><name sortKey="Josephs, O" uniqKey="Josephs O">O. Josephs</name></author><author><name sortKey="Rees, G" uniqKey="Rees G">G. Rees</name></author><author><name sortKey="Turner, R" uniqKey="Turner R">R. Turner</name></author><author><name sortKey="Frith, C D" uniqKey="Frith C">C. D. Frith</name></author><author><name sortKey="Friston, K J" uniqKey="Friston K">K. J. Friston</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Campbell, F A" uniqKey="Campbell F">F. A. Campbell</name></author><author><name sortKey="Goldman, B D" uniqKey="Goldman B">B. D. Goldman</name></author><author><name sortKey="Boccia, M L" uniqKey="Boccia M">M. L. Boccia</name></author><author><name sortKey="Skinner, M" uniqKey="Skinner M">M. Skinner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Cowan, N" uniqKey="Cowan N">N. Cowan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="De Valois, R L" uniqKey="De Valois R">R. L. De Valois</name></author><author><name sortKey="De Valois, K K" uniqKey="De Valois K">K. K. De Valois</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Farkas, D K" uniqKey="Farkas D">D. K. Farkas</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Fleming, M" uniqKey="Fleming M">M. Fleming</name></author><author><name sortKey="Levie, W H" uniqKey="Levie W">W. H. Levie</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Glimcher, P W" uniqKey="Glimcher P">P. W. Glimcher</name></author><author><name sortKey="Sparks, D L" uniqKey="Sparks D">D. L. Sparks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Grice, H P" uniqKey="Grice H">H. P. Grice</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gropper, G L" uniqKey="Gropper G">G. L. Gropper</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gunel, M" uniqKey="Gunel M">M. Gunel</name></author><author><name sortKey="Hand, B" uniqKey="Hand B">B. Hand</name></author><author><name sortKey="Gunduz, S" uniqKey="Gunduz S">S. Gunduz</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Helander, M" uniqKey="Helander M">M. Helander</name></author><author><name sortKey="Landauer, T" uniqKey="Landauer T">T. Landauer</name></author><author><name sortKey="Prabhu, P" uniqKey="Prabhu P">P. Prabhu</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Held, R" uniqKey="Held R">R. Held</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hu, A" uniqKey="Hu A">A. Hu</name></author><author><name sortKey="Wilson, T" uniqKey="Wilson T">T. Wilson</name></author><author><name sortKey="Ladak, H" uniqKey="Ladak H">H. Ladak</name></author><author><name sortKey="Haase, P" uniqKey="Haase P">P. Haase</name></author><author><name sortKey="Fung, K" uniqKey="Fung K">K. Fung</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Intriligator, J" uniqKey="Intriligator J">J. Intriligator</name></author><author><name sortKey="Cavanagh, P" uniqKey="Cavanagh P">P. Cavanagh</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Karns, T E" uniqKey="Karns T">T. E. Karns</name></author><author><name sortKey="Irvin, S J" uniqKey="Irvin S">S. J. Irvin</name></author><author><name sortKey="Suranic, S L" uniqKey="Suranic S">S. L. Suranic</name></author><author><name sortKey="Rivardo, M G" uniqKey="Rivardo M">M. G. Rivardo</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kosslyn, S M" uniqKey="Kosslyn S">S. M. Kosslyn</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kosslyn, S M" uniqKey="Kosslyn S">S. M. Kosslyn</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kosslyn, S M" uniqKey="Kosslyn S">S. M. Kosslyn</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Krauzlis, R J" uniqKey="Krauzlis R">R. J. Krauzlis</name></author><author><name sortKey="Liston, D" uniqKey="Liston D">D. Liston</name></author><author><name sortKey="Carello, C D" uniqKey="Carello C">C. D. Carello</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Larkin, J H" uniqKey="Larkin J">J. H. Larkin</name></author><author><name sortKey="Simon, H A" uniqKey="Simon H">H. A. Simon</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Levasseur, D G" uniqKey="Levasseur D">D. G. Levasseur</name></author><author><name sortKey="Sawyer, D K" uniqKey="Sawyer D">D. K. Sawyer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lund, A M" uniqKey="Lund A">A. M. Lund</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Macleod, C M" uniqKey="Macleod C">C. M. MacLeod</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Macmillan, N A" uniqKey="Macmillan N">N. A. Macmillan</name></author><author><name sortKey="Ornstein, A S" uniqKey="Ornstein A">A. S. Ornstein</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Mahar, S" uniqKey="Mahar S">S. Mahar</name></author><author><name sortKey="Yaylacicegi, U" uniqKey="Yaylacicegi U">U. Yaylacicegi</name></author><author><name sortKey="Janicki, T" uniqKey="Janicki T">T. Janicki</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Mejdal, S" uniqKey="Mejdal S">S. Mejdal</name></author><author><name sortKey="Mccauley, M E" uniqKey="Mccauley M">M. E. McCauley</name></author><author><name sortKey="Beringer, D B" uniqKey="Beringer D">D. B. Beringer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Osman, M E" uniqKey="Osman M">M. E. Osman</name></author><author><name sortKey="Hannafin, M J" uniqKey="Hannafin M">M. J. Hannafin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Palmer, S E" uniqKey="Palmer S">S. E. Palmer</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Parker, I" uniqKey="Parker I">I. Parker</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Posner, M I" uniqKey="Posner M">M. I. Posner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Potter, M C" uniqKey="Potter M">M. C. Potter</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Reisberg, D" uniqKey="Reisberg D">D. Reisberg</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rosch, E" uniqKey="Rosch E">E. Rosch</name></author><author><name sortKey="Simpson, C" uniqKey="Simpson C">C. Simpson</name></author><author><name sortKey="Miller, R S" uniqKey="Miller R">R. S. Miller</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Savoy, A" uniqKey="Savoy A">A. Savoy</name></author><author><name sortKey="Proctor, R W" uniqKey="Proctor R">R. W. Proctor</name></author><author><name sortKey="Salvendy, G" uniqKey="Salvendy G">G. Salvendy</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schmidt, S R" uniqKey="Schmidt S">S. R. Schmidt</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Schwartz, N H" uniqKey="Schwartz N">N. H. Schwartz</name></author><author><name sortKey="Ellsworth, L S" uniqKey="Ellsworth L">L. S. Ellsworth</name></author><author><name sortKey="Graham, L" uniqKey="Graham L">L. Graham</name></author><author><name sortKey="Knight, B" uniqKey="Knight B">B. Knight</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Shannon, C E" uniqKey="Shannon C">C. E. Shannon</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Shneiderman, B" uniqKey="Shneiderman B">B. Shneiderman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Smith, E E" uniqKey="Smith E">E. E. Smith</name></author><author><name sortKey="Kosslyn, S M" uniqKey="Kosslyn S">S. M. Kosslyn</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Smith, S" uniqKey="Smith S">S. Smith</name></author><author><name sortKey="Mosier, J" uniqKey="Mosier J">J. Mosier</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Speier, C" uniqKey="Speier C">C. Speier</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stanney, K" uniqKey="Stanney K">K. Stanney</name></author><author><name sortKey="Samman, S" uniqKey="Samman S">S. Samman</name></author><author><name sortKey="Reeves, L" uniqKey="Reeves L">L. Reeves</name></author><author><name sortKey="Hale, K" uniqKey="Hale K">K. Hale</name></author><author><name sortKey="Buff, W" uniqKey="Buff W">W. Buff</name></author><author><name sortKey="Bowers, C" uniqKey="Bowers C">C. Bowers</name></author><author><name sortKey="Goldiez, B" uniqKey="Goldiez B">B. Goldiez</name></author><author><name sortKey="Nicholson, D" uniqKey="Nicholson D">D. Nicholson</name></author><author><name sortKey="Lackey, S" uniqKey="Lackey S">S. Lackey</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stevens, S S" uniqKey="Stevens S">S. S. Stevens</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stoner, M R" uniqKey="Stoner M">M. R. Stoner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Stromeyer, C F" uniqKey="Stromeyer C">C. F. Stromeyer</name></author><author><name sortKey="Klein, S" uniqKey="Klein S">S. Klein</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Suchoff, I B" uniqKey="Suchoff I">I. B. Suchoff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Susskind, J E" uniqKey="Susskind J">J. E. Susskind</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Susskind, J E" uniqKey="Susskind J">J. E. Susskind</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sweller, J" uniqKey="Sweller J">J. Sweller</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Sweller, J" uniqKey="Sweller J">J. Sweller</name></author><author><name sortKey="Van Merrienboer, J J G" uniqKey="Van Merrienboer J">J. J. G. van Merrienboer</name></author><author><name sortKey="Paas, F G W C" uniqKey="Paas F">F. G. W. C. Paas</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Szabo, A" uniqKey="Szabo A">A. Szabo</name></author><author><name sortKey="Hastings, N" uniqKey="Hastings N">N. Hastings</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Taylor, J S" uniqKey="Taylor J">J. S. Taylor</name></author><author><name sortKey="Dube, C E" uniqKey="Dube C">C. E. Dube</name></author><author><name sortKey="Pipas, C F" uniqKey="Pipas C">C. F. Pipas</name></author><author><name sortKey="Fuller, B K" uniqKey="Fuller B">B. K. Fuller</name></author><author><name sortKey="Lavallee, L K" uniqKey="Lavallee L">L. K. Lavallee</name></author><author><name sortKey="Rosen, R" uniqKey="Rosen R">R. Rosen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Thomas, J P" uniqKey="Thomas J">J. P. Thomas</name></author><author><name sortKey="Gille, J" uniqKey="Gille J">J. Gille</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Travis, D" uniqKey="Travis D">D. Travis</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Tversky, B" uniqKey="Tversky B">B. Tversky</name></author><author><name sortKey="Morrsion, J B" uniqKey="Morrsion J">J. B. Morrsion</name></author><author><name sortKey="Betrancourt, M" uniqKey="Betrancourt M">M. Betrancourt</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ungerleider, L G" uniqKey="Ungerleider L">L. G. Ungerleider</name></author><author><name sortKey="Mishkin, M" uniqKey="Mishkin M">M. Mishkin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vekiri, I" uniqKey="Vekiri I">I. Vekiri</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vessey, I" uniqKey="Vessey I">I. Vessey</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wagoner, B" uniqKey="Wagoner B">B. Wagoner</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Watzman, S" uniqKey="Watzman S">S. Watzman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wittgenstein, L" uniqKey="Wittgenstein L">L. Wittgenstein</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wolfe, J M" uniqKey="Wolfe J">J. M. Wolfe</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wong, M S" uniqKey="Wong M">M. S. Wong</name></author><author><name sortKey="Lemaire, E D" uniqKey="Lemaire E">E. D. Lemaire</name></author><author><name sortKey="Leung, A K L" uniqKey="Leung A">A. K. L. Leung</name></author><author><name sortKey="Chan, M F" uniqKey="Chan M">M. F. Chan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Woodson, W" uniqKey="Woodson W">W. Woodson</name></author><author><name sortKey="Tillman, B" uniqKey="Tillman B">B. Tillman</name></author><author><name sortKey="Tillman, P" uniqKey="Tillman P">P. Tillman</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Front Psychol</journal-id><journal-id journal-id-type="iso-abbrev">Front Psychol</journal-id><journal-id journal-id-type="publisher-id">Front. Psychology</journal-id><journal-title-group><journal-title>Frontiers in Psychology</journal-title></journal-title-group><issn pub-type="epub">1664-1078</issn><publisher><publisher-name>Frontiers Research Foundation</publisher-name></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">22822402</article-id><article-id pub-id-type="pmc">3398435</article-id><article-id pub-id-type="doi">10.3389/fpsyg.2012.00230</article-id><article-categories><subj-group subj-group-type="heading"><subject>Psychology</subject><subj-group><subject>Original Research</subject></subj-group></subj-group></article-categories><title-group><article-title>PowerPoint<sup>®</sup> Presentation Flaws and Failures: A Psychological Analysis</article-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Kosslyn</surname><given-names>Stephen M.</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="author-notes" rid="fn001">*</xref></contrib><contrib contrib-type="author"><name><surname>Kievit</surname><given-names>Rogier A.</given-names></name><xref ref-type="aff" rid="aff2"><sup>2</sup></xref></contrib><contrib contrib-type="author"><name><surname>Russell</surname><given-names>Alexandra G.</given-names></name><xref ref-type="aff" rid="aff3"><sup>3</sup></xref></contrib><contrib contrib-type="author"><name><surname>Shephard</surname><given-names>Jennifer M.</given-names></name><xref ref-type="aff" rid="aff4"><sup>4</sup></xref></contrib></contrib-group><aff id="aff1"><sup>1</sup><institution>Center for Advanced Study in the Behavioral Sciences, Stanford University</institution><country>Stanford, CA USA</country></aff><aff id="aff2"><sup>2</sup><institution>Department of Psychology, University of Amsterdam</institution><country>Amsterdam, Netherlands</country></aff><aff id="aff3"><sup>3</sup><institution>Department of Psychology, Stanford University</institution><country>Stanford, CA, USA</country></aff><aff id="aff4"><sup>4</sup><institution>Division of Social Science, Harvard University</institution><country>Cambridge, MA, USA</country></aff><author-notes><fn fn-type="edited-by"><p>Edited by: Roger Azevedo, McGill University, Canada</p></fn><fn fn-type="edited-by"><p>Reviewed by: Ying Xie, Lewis University, USA; Marcus Lee Johnson, University of Cincinnati, USA</p></fn><corresp id="fn001">*Correspondence: Stephen M. Kosslyn, Center for Advanced Study in the Behavioral Sciences, Stanford University, 75 Alta Road, Stanford, CA 94305, USA. e-mail: <email xlink:type="simple">skosslyn@stanford.edu</email></corresp><fn fn-type="other" id="fn002"><p>This article was submitted to Frontiers in Educational Psychology, a specialty of Frontiers in Psychology.</p></fn></author-notes><pub-date pub-type="epreprint"><day>15</day><month>2</month><year>2012</year></pub-date><pub-date pub-type="epub"><day>17</day><month>7</month><year>2012</year></pub-date><pub-date pub-type="collection"><year>2012</year></pub-date><volume>3</volume><elocation-id>230</elocation-id><history><date date-type="received"><day>19</day><month>1</month><year>2012</year></date><date date-type="accepted"><day>19</day><month>6</month><year>2012</year></date></history><permissions><copyright-statement>Copyright © 2012 Kosslyn, Kievit, Russell and Shephard.</copyright-statement><copyright-year>2012</copyright-year><license license-type="open-access" xlink:href="http://www.frontiersin.org/licenseagreement"><license-p>This is an open-access article distributed under the terms of the <uri xlink:type="simple" xlink:href="http://creativecommons.org/licenses/by/3.0/">Creative Commons Attribution License</uri>, which permits use, distribution and reproduction in other forums, provided the original authors and source are credited and subject to any copyright notices concerning any third-party graphics etc.</license-p></license></permissions><abstract><p>Electronic slideshow presentations are often faulted anecdotally, but little empirical work has documented their faults. In Study 1 we found that eight psychological principles are often violated in PowerPoint<sup>®</sup> slideshows, and are violated to similar extents across different fields – for example, academic research slideshows generally were no better or worse than business slideshows. In Study 2 we found that respondents reported having noticed, and having been annoyed by, specific problems in presentations arising from violations of particular psychological principles. Finally, in Study 3 we showed that observers are not highly accurate in recognizing when particular slides violated a specific psychological rule. Furthermore, even when they correctly identified the violation, they often could not explain the nature of the problem. In sum, the psychological foundations for effective slideshow presentation design are neither obvious nor necessarily intuitive, and presentation designers in all fields, from education to business to government, could benefit from explicit instruction in relevant aspects of psychology.</p></abstract><kwd-group><kwd>PowerPoint<sup>®</sup></kwd><kwd>electronic slide show</kwd><kwd>presentation graphics</kwd><kwd>visual display design</kwd><kwd>clear communication</kwd><kwd>educational media</kwd><kwd>conveying information</kwd></kwd-group><counts><fig-count count="3"></fig-count><table-count count="7"></table-count><equation-count count="0"></equation-count><ref-count count="72"></ref-count><page-count count="22"></page-count><word-count count="17326"></word-count></counts></article-meta></front><body><sec><title>Introduction</title><p>PowerPoint<sup>®</sup>, Keynote<sup>®</sup>, and their electronic slideshow siblings have become pervasive means of communication. Indeed, in 2001, Microsoft estimated that an average of 30 million PowerPoint<sup>®</sup> presentations were given <italic>each day</italic> – and we can only imagine what that number is today (Parker, <xref ref-type="bibr" rid="B37">2001</xref>). Presentations using such computer programs are commonplace in academia, business, government, the military, and even K-12 schools. PowerPoint<sup>®</sup> has been used to teach subjects as varied as the three-dimensional structure of the larynx (Hu et al., <xref ref-type="bibr" rid="B21">2009</xref>), prosthetics and orthotics (Wong et al., <xref ref-type="bibr" rid="B71">2004</xref>), how to perform a testicular exam (Taylor et al., <xref ref-type="bibr" rid="B60">2004</xref>), developmental psychology (Susskind, <xref ref-type="bibr" rid="B56">2008</xref>), and physics (Gunel et al., <xref ref-type="bibr" rid="B18">2006</xref>).</p><p>Observers have offered varied opinions about PowerPoint<sup>®</sup> and the value of presentations that utilize it (e.g., for overviews, see Farkas, <xref ref-type="bibr" rid="B13">2006</xref>; Stoner, <xref ref-type="bibr" rid="B52">2007</xref>), and considerable empirical research has been conducted on the use of the medium (e.g., for reviews, see Susskind, <xref ref-type="bibr" rid="B55">2005</xref>, <xref ref-type="bibr" rid="B56">2008</xref>; Levasseur and Sawyer, <xref ref-type="bibr" rid="B29">2006</xref>; Savoy et al., <xref ref-type="bibr" rid="B42">2009</xref>). One set of this research has examined whether using this medium improves learning or comprehension. For example, Gunel et al. (<xref ref-type="bibr" rid="B18">2006</xref>) found that students learned more physics from PowerPoint<sup>®</sup> presentations than from chapter summaries, whereas other researchers have reported that students remember about the same amount of material following PowerPoint<sup>®</sup> presentations as they do following other media (such as overheads and use of the blackboard; e.g., Szabo and Hastings, <xref ref-type="bibr" rid="B59">2000</xref>; Beets and Lobingier, <xref ref-type="bibr" rid="B8">2001</xref>; Campbell et al., <xref ref-type="bibr" rid="B10">2004</xref>; Apperson et al., <xref ref-type="bibr" rid="B3">2006</xref>; Experiments 1 and 3; Susskind, <xref ref-type="bibr" rid="B55">2005</xref>, <xref ref-type="bibr" rid="B56">2008</xref>). Indeed, some studies find that PowerPoint<sup>®</sup> actually impairs learning. For example, Savoy et al. (<xref ref-type="bibr" rid="B42">2009</xref>) showed that students recalled more verbal information from a traditional lecture than they did when that information was presented verbally during a PowerPoint<sup>®</sup>-based lecture.</p><p>Researchers have also examined whether students prefer PowerPoint<sup>®</sup> presentations to other types of presentations. The results of such studies generally indicate that students do like PowerPoint<sup>®</sup> presentations (Susskind, <xref ref-type="bibr" rid="B56">2008</xref>; Savoy et al., <xref ref-type="bibr" rid="B42">2009</xref>). Apperson et al. (<xref ref-type="bibr" rid="B4">2008</xref>) asked psychology students what in particular they liked about PowerPoint<sup>®</sup> presentations, and found that they like to see lists built up one item at a time and that they also like outlines of key phrases, graphics, relevant sounds, and colored backgrounds. However, it is worth noting that Mahar et al. (<xref ref-type="bibr" rid="B33">2009</xref>) compared memory retention after an entire list had been presented simultaneously to retention when each item on the list had been presented individually, and found better memory (but only for two of the nine items on their test) in the simultaneous condition.</p><p>Even this brief overview of the literature reveals that the results are mixed. One reason for the varied results may lie in differences in the quality of the presentations: the medium is not the entire message; any medium can be used effectively or ineffectively. In our view, the key is not whether or not PowerPoint<sup>®</sup> is used, but rather <italic>how</italic> it is used. Nobody should be surprised if a PowerPoint<sup>®</sup> lecture utilizing poorly designed slides or presented by an inadequately prepared speaker is not effective. In this article, we use psychological principles to develop objective guidelines for slideshow design, slide preparation, and presentation execution, and use these guidelines to assess flaws in PowerPoint<sup>®</sup> slides and presentations.</p><p>In the three studies reported here, we empirically extend ideas presented in Kosslyn (<xref ref-type="bibr" rid="B25">2007</xref>, <xref ref-type="bibr" rid="B26">2010</xref>) and analyze how well PowerPoint<sup>®</sup> slideshows, slides, and presentations respect principles of human perception, memory, and comprehension. Specifically, we hypothesize that the psychological principles presented in the following section are violated in PowerPoint<sup>®</sup> slideshows across different fields (Study 1), that some types of presentation flaws will be noticeable and frustrating to audience members (Study 2), and that observers will have difficulties in identifying violations in the case of graphical displays in individual slides (Study 3).</p></sec><sec><title>Eight Cognitive Communication Principles</title><p>Levasseur and Sawyer (<xref ref-type="bibr" rid="B29">2006</xref>) note in their extensive review that remarkably little research provides direct guidelines for designing presentations (see also Suchoff, <xref ref-type="bibr" rid="B54">2006</xref>). However, much research on basic psychological processes has been conducted that can be used to formulate such guidelines. This observation has been appreciated by many researchers, who have understood the fundamental point that effective displays must play to the strengths of human information processing and must avoid relying on the weaknesses of such processing (e.g., Aspillaga, <xref ref-type="bibr" rid="B5">1996</xref>; Helander et al., <xref ref-type="bibr" rid="B19">1997</xref>; Mejdal et al., <xref ref-type="bibr" rid="B34">2001</xref>; Vekiri, <xref ref-type="bibr" rid="B65">2002</xref>; Watzman, <xref ref-type="bibr" rid="B68">2003</xref>; Stanney et al., <xref ref-type="bibr" rid="B50">2004</xref>).</p><p>We acknowledge that the following formulation is but one of several possible ways to organize these guidelines within the conventional view of human information processing, but this one (from Kosslyn, <xref ref-type="bibr" rid="B25">2007</xref>) appears to capture the major points of agreement among researchers. Moreover, this set of principles is a convenient way to organize the specific rules we examined, and it is these rules that identify the specific aspects of psychological processing we investigated. Thus, in addition to grouping the data according to the principles, we provide the data for each individual rule – which also allows readers to consider the results for those rules of particular interest.</p><p>In formulating our eight Cognitive Communication Principles, we began with a task analysis; that is, we considered what the viewer must do to understand a presentation. This analysis rests on the now-standard view that perception and comprehension can be decomposed into distinct classes of information processing operations (e.g., see Reisberg, <xref ref-type="bibr" rid="B40">2006</xref>; Smith and Kosslyn, <xref ref-type="bibr" rid="B47">2006</xref>), which underlie <italic>encoding</italic>, <italic>working memory</italic>, and <italic>accessing long-term memory</italic>. We focus here primarily on the visual modality because the visual aspects of the slide design are a major factor of PowerPoint<sup>®</sup> presentations (although the presenter’s speaking style and structuring of the presentation are also important, as we address in Study 2). In what follows, we first present the basic findings about information processing that led us to formulate a specific principle, and then explain the principle and how we operationalized it. Crucially, because all of these principles stem from known bottlenecks in information processing, slideshows, slides, and presentations that violate them will tax human information processing.</p><p>In what follows, although we organize the information processing operations into three categories (encoding, working memory, and accessing long-term memory), we do not mean to imply that these processes work in lockstep sequence. Rather, by focusing on the operations, we are led to define the bottlenecks that can disrupt presentations.</p><sec><title>Encoding processes</title><p>When viewing a sequence of slides, audience members first must encode what they see; if they fail to encode material, it may as well not exist. We formulated three principles based on essential facts about encoding.</p><sec><title>Discriminability</title><p>The initial step of encoding requires noticing the to-be-encoded material, which requires patterns to be clearly different from the background and from other patterns. This leads us to formulate <italic>the Principle of Discriminability</italic>: two properties (such as two colors, degrees of gray, or sizes) cannot convey different information unless they differ by a large enough proportion to be easily distinguished (Woodson et al., <xref ref-type="bibr" rid="B72">1992</xref>). This principle underlies camouflage, which occurs when two properties are so similar that they are not distinguishable – and there’s no place for camouflage in a presentation. In addition, we humans are not very good at discriminating among differences in the sizes of areas: we tend to underestimate areas, and do so to an increasingly large degree as the area increases (Stevens, <xref ref-type="bibr" rid="B51">1975</xref>). The Principle of Discriminability has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Unless typefaces are large enough, letters cannot easily be distinguished from each other.</p></list-item><list-item><label>(2)</label><p>If the color of text or graphics is not clearly distinct from the color of the background on which they appear, they cannot be readily distinguished.</p></list-item><list-item><label>(3)</label><p>Typefaces in which letters are similar (because they are visually complex, all upper case, all italics, or all bold) cannot be easily read.</p></list-item><list-item><label>(4)</label><p>Because patterns that are registered by separate “spatial frequency channels” are easily distinguished (Stromeyer and Klein, <xref ref-type="bibr" rid="B53">1974</xref>; De Valois and De Valois, <xref ref-type="bibr" rid="B12">1990</xref>), viewers can easily distinguish points that are about twice as thick as the lines that connect them and can easily distinguish texture patterns in which elements (e.g., stripes) vary by at least 2:1 (which will also avoid distracting “visual beats”).</p></list-item><list-item><label>(5)</label><p>Similarly, because orientations that differ by at least 30° are processed by different “orientation channels” (Thomas and Gille, <xref ref-type="bibr" rid="B61">1979</xref>; De Valois and De Valois, <xref ref-type="bibr" rid="B12">1990</xref>), viewers can easily distinguish orientations of the lines in different regions when they vary by at least 30°.</p></list-item><list-item><label>(6)</label><p>Finally, because of the way that different wavelengths of light focus on the retina, we have difficulty bringing cobalt blue (which is a mixture of red and blue) into focus, and we have difficulty focusing on red and blue at the same time (combinations of these colors tend to “shimmer”); thus text or other fine lines are difficult to distinguish if they are rendered in deep blues, and boundaries defined by adjacent red and blue regions are distracting.</p></list-item></list></sec><sec><title>Perceptual organization</title><p>Once material is noticed, processes that underlie figure/ground segregation organize the input into perceptual units that often correspond to objects (including words and graphics). This observation leads us to formulate the <italic>Principle of Perceptual Organization</italic>: people automatically organize elements into groups, which they then attend to and remember (Larkin and Simon, <xref ref-type="bibr" rid="B28">1987</xref>; Gropper, <xref ref-type="bibr" rid="B17">1991</xref>; Aspillaga, <xref ref-type="bibr" rid="B5">1996</xref>; Helander et al., <xref ref-type="bibr" rid="B19">1997</xref>; Vekiri, <xref ref-type="bibr" rid="B65">2002</xref>). Such groups emerge in numerous ways (Palmer, <xref ref-type="bibr" rid="B36">1992</xref>). Classical grouping “Laws” specify the perceptual conditions that lead viewers to see elements as a single group. For example, we tend to group together elements that are nearby (the so-called “Law of Proximity”). That is, we see “xxx xxx” as two groups of three each, not six separate x’s. Similarly, we tend to group together elements that appear similar (the so-called “Law of Similarity”). For instance, we see “///\\\” as two groups, not six separate lines.</p><p>Grouping also can be imposed by lines, such as those used in complex tables to define specific regions, which makes the tables easier to read. In addition, some visual dimensions are automatically grouped; these dimensions are “integral,” and cannot easily be considered independently. For example, height and width are not easily seen as distinct, but rather are organized into a single shape – and viewers register the overall area, not the two dimensions separately (Macmillan and Ornstein, <xref ref-type="bibr" rid="B32">1998</xref>). Other integral dimensions are hue, lightness, and saturation, which cannot easily be attended to independently; rather, our visual systems tend to group them into a single color – and thus these different dimensions of color cannot easily be used to convey separate measurements. The Principle of Perceptual Organization has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Labels initially are seen as applying to the closest graphic element.</p></list-item><list-item><label>(2)</label><p>Common color organizes parts of a display into a group, even if they are separated in space.</p></list-item><list-item><label>(3)</label><p>Common patterns or orderings organize parts of a display into a group, even if they are separated in space. For example, when the elements in a key (e.g., small squares of different colors, each associated with a label) are ordered the same way as the corresponding parts of the display (e.g., bars in a bar graph), the elements in the key will be grouped appropriately with the corresponding parts of the display.</p></list-item><list-item><label>(4)</label><p>Because shape and location largely are processed separately in the visual system (Ungerleider and Mishkin, <xref ref-type="bibr" rid="B64">1982</xref>), an element in one location will not always be easily grouped with elements in other locations. Hence, it is useful to construct a display so that separated-but-related elements are explicitly grouped (e.g., by using inner grid lines to group the tops of bars in a bar graph with locations along the <italic>Y</italic> axis at the left of the graph, if specific point values are to be read).</p></list-item><list-item><label>(5)</label><p>Comprehension is impaired by spurious groups, which form accidentally because of the grouping laws (e.g., such as those that can form when pairs of measurements for two or more categories are plotted in the same scatter plot, or when a banner at the top of a slide groups with specific objects in the display because of proximity or similarity).</p></list-item></list></sec><sec><title>Salience</title><p>Not all perceptual units can be processed simultaneously, if only because they are in different locations and the visual acuity of the eye is greatest at the fovea (and hence whatever people fixate their gazes on will be encoded with higher resolution). Thus, some units are encoded in detail before others. Attention selects which patterns will be processed in detail, and attention is automatically drawn to what is different; we immediately notice the nail that sticks above the floorboards or the one red light in a group of green lights. The <italic>Principle of Salience</italic> states: attention is drawn to large perceptible differences. In fact, a part of the brain (the superior colliculus) operates as an attentional reflex, automatically drawing our visual attention to large differences among stimuli (Posner, <xref ref-type="bibr" rid="B38">1980</xref>; Glimcher and Sparks, <xref ref-type="bibr" rid="B15">1992</xref>; Krauzlis et al., <xref ref-type="bibr" rid="B27">2004</xref>). The Principle of Salience has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Because movement is a particularly salient cue (Buchel et al., <xref ref-type="bibr" rid="B9">1998</xref>), animation captures and directs attention.</p></list-item><list-item><label>(2)</label><p>Text with a distinct format (color, size, or typeface) draws attention.</p></list-item><list-item><label>(3)</label><p>Visual disparities draw attention. For example, we notice a wedge exploded from a pie chart because the boundary has been disrupted; however, if more than one or two wedges are exploded, the boundary becomes so disrupted that the separated pieces are no longer salient.</p></list-item><list-item><label>(4)</label><p>If salience is aligned with importance (Helander et al., <xref ref-type="bibr" rid="B19">1997</xref>), the more important aspects of the slide (e.g., the title or topic sentence) or of an illustration (graph, diagram, demo) draw the audience members’ attention – which will also enhance later memory for those aspects (Schmidt, <xref ref-type="bibr" rid="B43">1991</xref>).</p></list-item><list-item><label>(5)</label><p>A special case of salience arises with colors: because of a quirk in how the corneas diffuse light and project it onto the retinas, “warm” colors (i.e., those with relatively long wavelengths, such as red) seem closer to the viewer than do “cool” colors (i.e., those with relatively short wavelengths, such as blue; see Held, <xref ref-type="bibr" rid="B20">1980</xref>; Allen and Rubin, <xref ref-type="bibr" rid="B1">1981</xref>; Travis, <xref ref-type="bibr" rid="B62">1991</xref>). Thus, if warmer colors are used for lines that pass beneath other lines, viewers will experience an annoying illusion in which material at the back appears to be struggling to move forward.</p></list-item></list></sec></sec><sec><title>Working memory</title><p>After visual patterns are encoded, they must be integrated. In a typical presentation, material must be integrated in working memory over the course of multiple slides, each of which may require multiple eye fixations. This integrating process is a prelude to fully comprehending both individual displays and the entire presentation. The high demands on working memory lead to the following two specific principles.</p><sec><title>Limited capacity</title><p>One of the key facts about working memory is that it has a very limited capacity (e.g., Baddeley, <xref ref-type="bibr" rid="B6">2007</xref>). The <italic>Principle of Limited Capacity</italic> states that people have a limited capacity to retain and to process information and will not understand a message if too much information must be retained or processed (Smith and Mosier, <xref ref-type="bibr" rid="B48">1986</xref>; Shneiderman, <xref ref-type="bibr" rid="B46">1992</xref>; Helander et al., <xref ref-type="bibr" rid="B19">1997</xref>; Lund, <xref ref-type="bibr" rid="B30">1997</xref>; Sweller et al., <xref ref-type="bibr" rid="B58">1998</xref>). The Principle of Limited Capacity has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>The amount of information that people can retain in working memory is defined in terms of psychological units, such as the perceptual groups produced by the classical perceptual grouping laws. We can hold in working memory only about four such units (Cowan, <xref ref-type="bibr" rid="B11">2001</xref>). However, each of these units in turn can comprise four units – and thus hierarchical organization can enhance our ability to hold information in mind.</p></list-item><list-item><label>(2)</label><p>In general, humans can track the movement of only about four units at the same time (Intriligator and Cavanagh, <xref ref-type="bibr" rid="B22">2001</xref> – although under special circumstances, more can be tracked; Alvarez and Franconeri, <xref ref-type="bibr" rid="B2">2007</xref>).</p></list-item><list-item><label>(3)</label><p>Eliminating the need to search for labels – for instance by directly labeling items in a display rather than using a key – reduces processing load (c.f. Sweller, <xref ref-type="bibr" rid="B57">1999</xref>).</p></list-item><list-item><label>(4)</label><p>Audience members need time to process the information that is presented.</p></list-item><list-item><label>(5)</label><p>However, conversely, if slides fade-in or fade-out very slowly, the audience members may organize them incorrectly and then have to break their initial organization – which also requires effort (Potter, <xref ref-type="bibr" rid="B39">1966</xref>).</p></list-item></list></sec><sec><title>Informative change</title><p>Because working memory has limited capacity, extraneous information can easily overwhelm it at the expense of relevant information. But, by the same token, visual or auditory cues can be used to help organize the information. According to the <italic>Principle of Informative Change</italic>, people expect changes in perceptual properties to carry information, and expect every necessary piece of information to be conveyed by such a perceptible change. Indeed, the very concept of “information” has been defined in terms of change: only when there is a change is information conveyed (Shannon, <xref ref-type="bibr" rid="B45">1948</xref>). The Principle of Informative Change has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Audience members assume that words, graphics, or other changes in appearance convey new information (Smith and Mosier, <xref ref-type="bibr" rid="B48">1986</xref>; Shneiderman, <xref ref-type="bibr" rid="B46">1992</xref>; Aspillaga, <xref ref-type="bibr" rid="B5">1996</xref>; Helander et al., <xref ref-type="bibr" rid="B19">1997</xref>; Lund, <xref ref-type="bibr" rid="B30">1997</xref>). For example, audience members assume that new information is being conveyed by changes in the appearance of the background, bullet points, or color or typeface of text. Random or arbitrary changes in appearance, in transitions between slides, or in terminology (“fowl” versus “bird”) can lead the audience astray.</p></list-item><list-item><label>(2)</label><p>Clearly marking the beginnings and ends of sections of a presentation (for instance by presenting a title or concluding slide with a distinct format, typeface, or background) helps audience members follow the presentation.</p></list-item></list></sec></sec><sec><title>Accessing long-term memory</title><p>Our task analysis leads us to posit a third class of relevant processes. Encoding information and integrating it appropriately would be useless if the meaning of the material were not extracted. In order to ascribe meaning to stimuli, one must compare them with material previously stored in long-term memory; it is only by retrieving associated information that we comprehend the import of what we see. The following principles address factors that affect the ease of accessing long-term memory and activating the relevant stored information.</p><sec><title>Appropriate knowledge</title><p>Meaning can be ascribed to a pattern only if the person has the requisite information already stored in long-term memory; to reach an audience, the presenter must make contact with what the audience members already know (Osman and Hannafin, <xref ref-type="bibr" rid="B35">1992</xref>; Shneiderman, <xref ref-type="bibr" rid="B46">1992</xref>; Fleming and Levie, <xref ref-type="bibr" rid="B14">1993</xref>; Lund, <xref ref-type="bibr" rid="B30">1997</xref>; Schwartz et al., <xref ref-type="bibr" rid="B44">1998</xref>). The <italic>Principle of Appropriate Knowledge</italic> states: communication requires prior knowledge of relevant concepts, jargon, and symbols. If the presenter relies on novel concepts, jargon, or symbols, the audience members will fail to understand. The Principle of Appropriate Knowledge has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Unfamiliar (for that audience) concepts, conventions, formats, terminology, and symbols may not be understood. Moreover, when they are understood, they will likely require effortful processing (which will be accomplished only if the audience is highly motivated).</p></list-item><list-item><label>(2)</label><p>If unfamiliar (for that audience) concepts, conventions, formats, terminology, and symbols are absolutely necessary to convey the message, they must be explicitly introduced and explained – and such explanations are more effective if they draw on information that is familiar to the audience.</p></list-item></list></sec><sec><title>Compatibility</title><p>The meaning of a stimulus will be difficult to extract if the interpretation of its surface properties (such as the size or color) is inconsistent with its symbolic meaning. The <italic>Principle of Compatibility</italic> states that a message is easiest to understand if its form is compatible with its meaning (Vessey, <xref ref-type="bibr" rid="B66">1991</xref>; Woodson et al., <xref ref-type="bibr" rid="B72">1992</xref>; Vekiri, <xref ref-type="bibr" rid="B65">2002</xref>; Speier, <xref ref-type="bibr" rid="B49">2006</xref>). This principle is perhaps most evident when it is breached, as demonstrated by the classic “Stroop effect” (MacLeod, <xref ref-type="bibr" rid="B31">1991</xref>). In the Stroop effect, people have more difficulty naming the color of the ink used to print words when the words name a different color from the ink (e.g., blue ink used to print the word “red”) than when words name the same color (e.g., blue ink used to print the word “blue”). We register both the surface properties (e.g., the color of the ink) and the meaning. This principle applies across perceptual modalities, and thus comprehension generally is best when audio and visual contents coordinate with text and the overall message that is being conveyed; inappropriate sounds and visuals will interfere with comprehension. The Principle of Compatibility has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>Because hue is a metathetic (values on metathetic dimensions are arranged qualitatively) variable, variations in hue are not seen automatically as signaling different amounts of a quantity; in contrast, because saturation and intensity (or brightness) are prothetic variables (values on prothetic dimensions are arranged quantitatively), variations along these dimensions do line up with variations in quantities (Stevens, <xref ref-type="bibr" rid="B51">1975</xref>).</p></list-item><list-item><label>(2)</label><p>Animation interferes with comprehension if it does not fit the natural movements of the object (e.g., a picture of a car should not drop down from the top), and sounds and slide backgrounds will interfere if they are not appropriate for the topic (a floral background, or the sounds of birds chirping, are not compatible with a presentation about carbon reservoirs in the ocean).</p></list-item><list-item><label>(3)</label><p>Viewers most easily interpret icons that depict the typical examples of represented items. For example, a picture of a duck effectively illustrates “water fowl” but not “pet bird,” and vice versa for a picture of a canary (Rosch et al., <xref ref-type="bibr" rid="B41">1976</xref>).</p></list-item><list-item><label>(4)</label><p>Old-fashioned looking typeface would send a conflicting message if used in a written description of a high-tech device.</p></list-item><list-item><label>(5)</label><p>Because they make different sorts of information explicit, different sorts of graphics are appropriate for making different points (Tversky et al., <xref ref-type="bibr" rid="B63">2002</xref>). Line graphs (rather than bar or mixed graphs) illustrate trends effectively because the continuous variation in the height of a line in a graph directly indicates the continuous variations of a measurement. Similarly, crossing lines in a graph indicate interactions more effectively than sets of bars. Bar graphs illustrate specific values (not trends) effectively because the discrete heights of the bars directly indicate specific measurements. Maps illustrate complex information about geographic territories or show alternate routes to a destination. Charts effectively illustrate organizational structure, a sequence of steps, or processes over time (“flow charts”).</p></list-item></list></sec><sec><title>Relevance</title><p>Finally, the message must be calibrated so that neither too much nor too little information is presented for a specific audience (Grice, <xref ref-type="bibr" rid="B16">1975</xref>). (Note: the Principle of Informative Change states that, given a specific message, the requisite information must be provided – the present point is about providing an appropriate message in the first place.) It is clear that not providing enough information is a problem, but perhaps it is less immediately evident that providing too much information (such as extraneous graphics, text, or audio) is also a problem. Presenting too much information is a problem in part because this forces viewers to search for the relevant information, which requires effort (e.g., Wolfe, <xref ref-type="bibr" rid="B70">1998</xref>). The <italic>Principle of Relevance</italic> states that communication is most effective when neither too much nor too little information is presented (Smith and Mosier, <xref ref-type="bibr" rid="B48">1986</xref>; Woodson et al., <xref ref-type="bibr" rid="B72">1992</xref>; Vekiri, <xref ref-type="bibr" rid="B65">2002</xref>; Bartsch and Cobern, <xref ref-type="bibr" rid="B7">2003</xref>). This principle has the following corollaries:</p><list list-type="simple"><list-item><label>(1)</label><p>To decide what is too much or too little, one must know about the nature of the message: depending on what the intended point is, specific information can be necessary or extraneous.</p></list-item><list-item><label>(2)</label><p>When attempting to understand information, people (largely unconsciously) organize it into a narrative (c.f. Wagoner, <xref ref-type="bibr" rid="B67">2008</xref>; Karns et al., <xref ref-type="bibr" rid="B23">2009</xref>). Defining the topic and presenting a roadmap at the outset (in an outline or other overview) facilitates this process.</p></list-item><list-item><label>(3)</label><p>Graphics (photos, drawings, graphs, diagrams), audio, and video can provide detail to illustrate the relevant concepts clearly. However, pictures are often ambiguous (Wittgenstein, <xref ref-type="bibr" rid="B69">1953/2001</xref>), and when they are ambiguous labels can clarify them.</p></list-item></list><p>In the following study we use these Cognitive Communication Principles, and the specific rules that grow out of them, to evaluate a wide variety of slideshows and slides. We ask the following questions: first, are violations of the principles common? Given that many corollaries of these principles are not intuitively obvious, we hypothesize that we will find many violations. Second, are the violations equally common in different fields? Because human beings are preparing the slides in all cases, we have no grounds for hypothesizing differences in the frequency of violations in the different fields.</p></sec></sec></sec><sec><title>Study 1</title><p>Levasseur and Sawyer (<xref ref-type="bibr" rid="B29">2006</xref>) noted that there is remarkably little research on how slides should be designed so that they function effectively. One possible reason for this dearth is that many may feel that “good design” is intuitively clear to most people, and hence there is no reason to study it. Consistent with this conjecture, some have claimed that the sorts of psychological principles just discussed are obvious, and rarely if ever would be violated in a presentation. To evaluate this supposition, we examined a sample of PowerPoint<sup>®</sup> slideshows. Specifically, we used a stratified sampling procedure to examine a selection of slideshows in five categories: Research (academic), Education, Government, Business, and Miscellaneous. We formulated 137 ways (each specified in a rule) in which the eight principles just summarized could be violated (see Table <xref ref-type="table" rid="T1">1</xref>).</p><table-wrap id="T1" position="float"><label>Table 1</label><caption><p><bold>List of rules for each principle and the proportion of presentations violating each rule (according to strict and liberal criteria) for Study 1</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1">Violations (Strict)</th><th align="left" rowspan="1" colspan="1">Violations (Liberal)</th></tr></thead><tbody><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>APPROPRIATE KNOWLEDGE</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">Unusual bullet symbols are used</td><td align="left" rowspan="1" colspan="1">0.379</td><td align="left" rowspan="1" colspan="1">0.257</td></tr><tr><td align="left" rowspan="1" colspan="1">Non-standard or unfamiliar display formats are used</td><td align="left" rowspan="1" colspan="1">0.343</td><td align="left" rowspan="1" colspan="1">0.300</td></tr><tr><td align="left" rowspan="1" colspan="1">The key is not at the top right of a single panel or centered over multiple panels</td><td align="left" rowspan="1" colspan="1">0.279</td><td align="left" rowspan="1" colspan="1">0.264</td></tr><tr><td align="left" rowspan="1" colspan="1">The title is not at the top of the slide</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.057</td></tr><tr><td align="left" rowspan="1" colspan="1">Symbols are potentially ambiguous for the audience</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Standard conventions for fonts are not used</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Terms do not convey the appropriate denotations and connotations</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>COMPATIBILITY</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">The layout of a chart is not compatible with the subject matter</td><td align="left" rowspan="1" colspan="1">0.107</td><td align="left" rowspan="1" colspan="1">0.079</td></tr><tr><td align="left" rowspan="1" colspan="1">A line graph is not used to display trends</td><td align="left" rowspan="1" colspan="1">0.093</td><td align="left" rowspan="1" colspan="1">0.093</td></tr><tr><td align="left" rowspan="1" colspan="1">A line graph is not used when the <italic>X</italic> axis uses a continuous scale</td><td align="left" rowspan="1" colspan="1">0.086</td><td align="left" rowspan="1" colspan="1">0.086</td></tr><tr><td align="left" rowspan="1" colspan="1">Font is incompatible with its connotations (sans serif implies modern, technological; serif implies traditional)</td><td align="left" rowspan="1" colspan="1">0.064</td><td align="left" rowspan="1" colspan="1">0.036</td></tr><tr><td align="left" rowspan="1" colspan="1">The background pattern is inappropriate to the main point of the display</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Sounds are not appropriate for the topic and point being made</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">More inclusive categories are not higher in an organizational chart</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">The style of photos or clipart is not compatible with the message</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Sounds, text, and graphics are not coordinated</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">A bar graph is not used to illustrate differences between specific point values</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Animations/videos are not compatible with the represented object or event</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Saturation and lightness are not varied for hues that indicate greater amounts (which is a problem because variations in hue alone are not naturally seen as corresponding to variations in amount)</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">A line graph is not used to display interactions over two levels on the <italic>X</italic> axis</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">A bar graph is not used when more than two values are on an <italic>X</italic> axis that does not show a continuous scale</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">A chart is not used to illustrate sequences of steps over time</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">A map is not presented when more than one route is possible</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Mixed bar/line displays are used to show interactions</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Colors are incompatible (given common conventions) with the meaning of the colored elements</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>DISCRIMINABILITY</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">All uppercase, all italics, or all bold typefaces are used</td><td align="left" rowspan="1" colspan="1">0.807</td><td align="left" rowspan="1" colspan="1">0.557</td></tr><tr><td align="left" rowspan="1" colspan="1">Words are not large enough (i.e., greater than 20 point) to be easily read</td><td align="left" rowspan="1" colspan="1">0.664</td><td align="left" rowspan="1" colspan="1">0.650</td></tr><tr><td align="left" rowspan="1" colspan="1">Deep, heavily saturated blue is used for text or graphics</td><td align="left" rowspan="1" colspan="1">0.550</td><td align="left" rowspan="1" colspan="1">0.429</td></tr><tr><td align="left" rowspan="1" colspan="1">Entries in a table are too small to be read easily</td><td align="left" rowspan="1" colspan="1">0.514</td><td align="left" rowspan="1" colspan="1">0.486</td></tr><tr><td align="left" rowspan="1" colspan="1">Underlining is used</td><td align="left" rowspan="1" colspan="1">0.464</td><td align="left" rowspan="1" colspan="1">0.257</td></tr><tr><td align="left" rowspan="1" colspan="1">Colors shimmer</td><td align="left" rowspan="1" colspan="1">0.421</td><td align="left" rowspan="1" colspan="1">0.393</td></tr><tr><td align="left" rowspan="1" colspan="1">Photos and clipart become too grainy when inserted into the slide</td><td align="left" rowspan="1" colspan="1">0.414</td><td align="left" rowspan="1" colspan="1">0.314</td></tr><tr><td align="left" rowspan="1" colspan="1">Red and blue are used in adjacent regions</td><td align="left" rowspan="1" colspan="1">0.357</td><td align="left" rowspan="1" colspan="1">0.236</td></tr><tr><td align="left" rowspan="1" colspan="1">Information-conveying visual properties are not discriminable</td><td align="left" rowspan="1" colspan="1">0.129</td><td align="left" rowspan="1" colspan="1">0.129</td></tr><tr><td align="left" rowspan="1" colspan="1">Text cannot be easily discriminated from the background</td><td align="left" rowspan="1" colspan="1">0.129</td><td align="left" rowspan="1" colspan="1">0.129</td></tr><tr><td align="left" rowspan="1" colspan="1">Hues are not well separated in the spectrum</td><td align="left" rowspan="1" colspan="1">0.114</td><td align="left" rowspan="1" colspan="1">0.114</td></tr><tr><td align="left" rowspan="1" colspan="1">Different lines connect different points, but the lines are not easily discriminated</td><td align="left" rowspan="1" colspan="1">0.064</td><td align="left" rowspan="1" colspan="1">0.064</td></tr><tr><td align="left" rowspan="1" colspan="1">The foreground and background are not easily discriminable</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Labels and patches in a key are difficult to tell apart</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Points or symbols connected by different lines are not easily discriminated</td><td align="left" rowspan="1" colspan="1">0.043</td><td align="left" rowspan="1" colspan="1">0.043</td></tr><tr><td align="left" rowspan="1" colspan="1">If lines connect discrete points, the points are not at least twice as thick as the line</td><td align="left" rowspan="1" colspan="1">0.043</td><td align="left" rowspan="1" colspan="1">0.043</td></tr><tr><td align="left" rowspan="1" colspan="1">Adjacent colors have similar lightness</td><td align="left" rowspan="1" colspan="1">0.036</td><td align="left" rowspan="1" colspan="1">0.029</td></tr><tr><td align="left" rowspan="1" colspan="1">Visual beats occur</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Visually complex fonts are used</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Double-spaced bullets are used</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Dashes in lines do not differ by at least 2–1</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Sounds are not high fidelity</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Orientation of textures used to fill patterns does not vary by at least 30° of arc</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Spacing of texture patterns with similar orientations does not vary by a ratio of at least 2–1</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Area of elements is used to convey precise quantities</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>INFORMATIVE CHANGES</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">Visual or auditory characteristics change even when they do not signal a change in information</td><td align="left" rowspan="1" colspan="1">0.721</td><td align="left" rowspan="1" colspan="1">0.564</td></tr><tr><td align="left" rowspan="1" colspan="1">There is no crisp ending to signal that the presentation, or a given part, is over</td><td align="left" rowspan="1" colspan="1">0.643</td><td align="left" rowspan="1" colspan="1">0.593</td></tr><tr><td align="left" rowspan="1" colspan="1">Serif and sans serif are mixed arbitrarily</td><td align="left" rowspan="1" colspan="1">0.371</td><td align="left" rowspan="1" colspan="1">0.207</td></tr><tr><td align="left" rowspan="1" colspan="1">A consistent and distinctively formatted slide does not signal the beginning of each new part/group of the presentation</td><td align="left" rowspan="1" colspan="1">0.314</td><td align="left" rowspan="1" colspan="1">0.300</td></tr><tr><td align="left" rowspan="1" colspan="1">Different bullet symbols are used for entries in a list of similar items</td><td align="left" rowspan="1" colspan="1">0.257</td><td align="left" rowspan="1" colspan="1">0.186</td></tr><tr><td align="left" rowspan="1" colspan="1">Different transitions are used randomly for different slides</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.057</td></tr><tr><td align="left" rowspan="1" colspan="1">The same terminology is not used in labels and surrounding text</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Tonal quality or volume is varied randomly</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>LIMITED CAPACITY</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">0.957</td><td align="left" rowspan="1" colspan="1">0.943</td></tr><tr><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">0.914</td><td align="left" rowspan="1" colspan="1">0.871</td></tr><tr><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.914</td><td align="left" rowspan="1" colspan="1">0.879</td></tr><tr><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.857</td><td align="left" rowspan="1" colspan="1">0.836</td></tr><tr><td align="left" rowspan="1" colspan="1">Slides contain more than what can be read aloud in about 1 min</td><td align="left" rowspan="1" colspan="1">0.314</td><td align="left" rowspan="1" colspan="1">0.314</td></tr><tr><td align="left" rowspan="1" colspan="1">Complex displays are not built up a part at a time</td><td align="left" rowspan="1" colspan="1">0.271</td><td align="left" rowspan="1" colspan="1">0.264</td></tr><tr><td align="left" rowspan="1" colspan="1">Viewers are expected to read a complex table</td><td align="left" rowspan="1" colspan="1">0.186</td><td align="left" rowspan="1" colspan="1">0.164</td></tr><tr><td align="left" rowspan="1" colspan="1">More than four perceptual units are presented in one panel of a graphic</td><td align="left" rowspan="1" colspan="1">0.164</td><td align="left" rowspan="1" colspan="1">0.164</td></tr><tr><td align="left" rowspan="1" colspan="1">The conceptual structure or outline is not organized hierarchically into groups of no more than four elements</td><td align="left" rowspan="1" colspan="1">0.093</td><td align="left" rowspan="1" colspan="1">0.093</td></tr><tr><td align="left" rowspan="1" colspan="1">Slides fade-in or fade-out too slowly</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.036</td></tr><tr><td align="left" rowspan="1" colspan="1">Content elements are not labeled directly whenever space permits</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">A key is used when direct labels could be used instead</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">More than four separate perceptual groups are moved simultaneously</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Hierarchical labeling of graphics is not used</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Multiple pie graphs are used to compare corresponding parts even though the proportions vary greatly</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>PERCEPTUAL ORGANIZATION</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">In tables with more than two rows and two columns, grid lines are not included</td><td align="left" rowspan="1" colspan="1">0.150</td><td align="left" rowspan="1" colspan="1">0.150</td></tr><tr><td align="left" rowspan="1" colspan="1">The title is too close to other words or patterns and groups with them</td><td align="left" rowspan="1" colspan="1">0.100</td><td align="left" rowspan="1" colspan="1">0.086</td></tr><tr><td align="left" rowspan="1" colspan="1">Parts of background patterns group with parts of the foreground</td><td align="left" rowspan="1" colspan="1">0.079</td><td align="left" rowspan="1" colspan="1">0.079</td></tr><tr><td align="left" rowspan="1" colspan="1">In a key, labels and patches fail to group together</td><td align="left" rowspan="1" colspan="1">0.079</td><td align="left" rowspan="1" colspan="1">0.071</td></tr><tr><td align="left" rowspan="1" colspan="1">The space between bar clusters is less than the width of two bars</td><td align="left" rowspan="1" colspan="1">0.064</td><td align="left" rowspan="1" colspan="1">0.057</td></tr><tr><td align="left" rowspan="1" colspan="1">Patches in keys and their corresponding content elements are in different orders</td><td align="left" rowspan="1" colspan="1">0.064</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Labels are not grouped with the appropriate elements of the display</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.057</td></tr><tr><td align="left" rowspan="1" colspan="1">A banner at the top is not clearly distinct from the other material</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">An inner grid is not used in a graph when precise values are important</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Corresponding bars are not arranged in the same way</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Words in the same label are not close together and typographically similar</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Hue, lightness, and saturation specify different measurements (which is a problem because we cannot easily see these properties as independent)</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Portions of the same text, line, or graphic move separately</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Height and width are used to specify separate variables</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">There is no space between bar clusters</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Corresponding bars are not marked in the same way</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>RELEVANCE</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">Bullets do not introduce topic sentences/phrases or specific cases</td><td align="left" rowspan="1" colspan="1">0.471</td><td align="left" rowspan="1" colspan="1">0.471</td></tr><tr><td align="left" rowspan="1" colspan="1">Either more or less detail than required for the point is presented</td><td align="left" rowspan="1" colspan="1">0.379</td><td align="left" rowspan="1" colspan="1">0.379</td></tr><tr><td align="left" rowspan="1" colspan="1"><italic>X</italic> and <italic>Y</italic> axes are not clearly identifiable and appropriately labeled</td><td align="left" rowspan="1" colspan="1">0.207</td><td align="left" rowspan="1" colspan="1">0.207</td></tr><tr><td align="left" rowspan="1" colspan="1">Problem, question, or topic of the presentation is not defined</td><td align="left" rowspan="1" colspan="1">0.150</td><td align="left" rowspan="1" colspan="1">0.121</td></tr><tr><td align="left" rowspan="1" colspan="1">Tables show more than the information needed to make the point</td><td align="left" rowspan="1" colspan="1">0.136</td><td align="left" rowspan="1" colspan="1">0.136</td></tr><tr><td align="left" rowspan="1" colspan="1">Gratuitous animation, which obscures rather than illuminates the point, is presented</td><td align="left" rowspan="1" colspan="1">0.121</td><td align="left" rowspan="1" colspan="1">0.079</td></tr><tr><td align="left" rowspan="1" colspan="1">Every graphic or table, as well as each component of the content material, is not labeled (unless the identity is self-evident)</td><td align="left" rowspan="1" colspan="1">0.086</td><td align="left" rowspan="1" colspan="1">0.079</td></tr><tr><td align="left" rowspan="1" colspan="1">Gratuitous graphics, videos, or sounds are presented</td><td align="left" rowspan="1" colspan="1">0.071</td><td align="left" rowspan="1" colspan="1">0.064</td></tr><tr><td align="left" rowspan="1" colspan="1">Photos or clipart are named with a word or phrase that does not bear directly on the point</td><td align="left" rowspan="1" colspan="1">0.057</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Photos and clipart do not: define the context, introduce an abstract idea, or evoke a specific emotion</td><td align="left" rowspan="1" colspan="1">0.050</td><td align="left" rowspan="1" colspan="1">0.036</td></tr><tr><td align="left" rowspan="1" colspan="1">An overview of a list is not presented</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.029</td></tr><tr><td align="left" rowspan="1" colspan="1">Complex concepts are not illustrated clearly with graphics (displays, videos, sounds, or animations)</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.029</td></tr><tr><td align="left" rowspan="1" colspan="1">If sounds are presented, they do not provide useful and pertinent information</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">Sounds are used without explanation or labeling</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">Bars extend beyond the end of the scale</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">A table is not presented when needed (i.e., when specific values are important)</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Labeled routes in a map are not important, or important routes are not labeled</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Labels are not provided when precise amounts are relevant</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">Important distances are not labeled directly</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>SALIENCE</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">Different colors are not being used for emphasis or to specify</td><td align="left" rowspan="1" colspan="1">0.329</td><td align="left" rowspan="1" colspan="1">0.314</td></tr><tr><td align="left" rowspan="1" colspan="1">The most important content element is not the most salient</td><td align="left" rowspan="1" colspan="1">0.321</td><td align="left" rowspan="1" colspan="1">0.314</td></tr><tr><td align="left" rowspan="1" colspan="1">The color of the text is more salient than the color of the title</td><td align="left" rowspan="1" colspan="1">0.286</td><td align="left" rowspan="1" colspan="1">0.243</td></tr><tr><td align="left" rowspan="1" colspan="1">Color makes less important elements salient</td><td align="left" rowspan="1" colspan="1">0.243</td><td align="left" rowspan="1" colspan="1">0.236</td></tr><tr><td align="left" rowspan="1" colspan="1">The salience of lines or bars does not reflect relative importance</td><td align="left" rowspan="1" colspan="1">0.136</td><td align="left" rowspan="1" colspan="1">0.129</td></tr><tr><td align="left" rowspan="1" colspan="1">The background pattern is very salient</td><td align="left" rowspan="1" colspan="1">0.100</td><td align="left" rowspan="1" colspan="1">0.093</td></tr><tr><td align="left" rowspan="1" colspan="1">The title is not typographically distinct</td><td align="left" rowspan="1" colspan="1">0.093</td><td align="left" rowspan="1" colspan="1">0.093</td></tr><tr><td align="left" rowspan="1" colspan="1">Illustrations do not face the center of the slide (and hence direct attention to the side)</td><td align="left" rowspan="1" colspan="1">0.064</td><td align="left" rowspan="1" colspan="1">0.064</td></tr><tr><td align="left" rowspan="1" colspan="1">Warm colors do not define the foreground</td><td align="left" rowspan="1" colspan="1">0.043</td><td align="left" rowspan="1" colspan="1">0.043</td></tr><tr><td align="left" rowspan="1" colspan="1">More salient labels are not used to label more important components of the display</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.029</td></tr><tr><td align="left" rowspan="1" colspan="1">More than 25% of the wedges in a pie are exploded</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.029</td></tr><tr><td align="left" rowspan="1" colspan="1">Sounds do not grab viewers’ attention appropriately</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td colspan="3" style="background-color:gray" align="left" rowspan="1"><bold>“OVER-DETERMINED”</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">The title of a very complex slide is not presented before presenting the content elements</td><td align="left" rowspan="1" colspan="1">0.171</td><td align="left" rowspan="1" colspan="1">0.164</td></tr><tr><td align="left" rowspan="1" colspan="1">The title of a slide does not focus attention on the most important point</td><td align="left" rowspan="1" colspan="1">0.093</td><td align="left" rowspan="1" colspan="1">0.071</td></tr><tr><td align="left" rowspan="1" colspan="1">Irrelevant words or graphics are easily distinguishable from the background</td><td align="left" rowspan="1" colspan="1">0.093</td><td align="left" rowspan="1" colspan="1">0.079</td></tr><tr><td align="left" rowspan="1" colspan="1">Different types of data are graphed in a single display even when they are unrelated</td><td align="left" rowspan="1" colspan="1">0.071</td><td align="left" rowspan="1" colspan="1">0.064</td></tr><tr><td align="left" rowspan="1" colspan="1">Pictures and icons used as labels do not evoke the appropriate concepts</td><td align="left" rowspan="1" colspan="1">0.050</td><td align="left" rowspan="1" colspan="1">0.050</td></tr><tr><td align="left" rowspan="1" colspan="1">Wedges in a pie graph are not arranged in a simple progression</td><td align="left" rowspan="1" colspan="1">0.043</td><td align="left" rowspan="1" colspan="1">0.043</td></tr><tr><td align="left" rowspan="1" colspan="1">Shapes of meaningful regions are not easily identifiable</td><td align="left" rowspan="1" colspan="1">0.036</td><td align="left" rowspan="1" colspan="1">0.036</td></tr><tr><td align="left" rowspan="1" colspan="1">The same size and font is not used for labels of corresponding components</td><td align="left" rowspan="1" colspan="1">0.029</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">A map is not used to label complex sets of information about a territory</td><td align="left" rowspan="1" colspan="1">0.021</td><td align="left" rowspan="1" colspan="1">0.021</td></tr><tr><td align="left" rowspan="1" colspan="1">A chart is not used to convey overall organizational structure</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Viewers must read moving words</td><td align="left" rowspan="1" colspan="1">0.014</td><td align="left" rowspan="1" colspan="1">0.014</td></tr><tr><td align="left" rowspan="1" colspan="1">Multiple panels are not being used to highlight specific comparisons</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Pairs of measurements for more than one category are shown in a scatterplot</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">A graph is not used to illustrate relative amounts</td><td align="left" rowspan="1" colspan="1">0.007</td><td align="left" rowspan="1" colspan="1">0.007</td></tr><tr><td align="left" rowspan="1" colspan="1">Animation is absent when it could be used to direct attention to a complex topic</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">A horizontal bar graph is not used when labels are too long to fit under a vertical display</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr><tr><td align="left" rowspan="1" colspan="1">All parts of static 3D diagrams are not shown from the same viewpoint</td><td align="left" rowspan="1" colspan="1">0.000</td><td align="left" rowspan="1" colspan="1">0.000</td></tr></tbody></table></table-wrap><sec sec-type="materials|methods"><title>Materials and methods</title><p>For all searching and coding, Mac OSX and Microsoft Excel were used. We used the Google search engine for the sampling procedure.</p><p>We first acquired a random, stratified sample of PowerPoint<sup>®</sup> slideshows from the web. Following this, we asked two judges to score each PowerPoint<sup>®</sup> slideshow independently, using a checklist (see Table <xref ref-type="table" rid="T1">1</xref>) to search for violations of specific rules of effective PowerPoint<sup>®</sup> communication; these rules were special cases of the eight general principles discussed in the Introduction. As noted below, two additional judges evaluated the slideshows using slightly different criteria.</p><sec><title>Search procedure</title><p>Before we present the results of our sampling, we need to explain how we produced the sample. To ensure that there was no potential for bias (that is, sampling non-representative slideshows), we randomly selected slideshows within each category. To this end, we implemented a two-step procedure. We first constructed a list of keywords for each of four categories – Research (academic), Education, Government, and Business – by searching an electronic resource (e.g., EbscoHost) for the category name and then noting the keywords of the first few academic articles to appear in the search results. We then randomly paired each keyword with two numbers, which ranged from 1 to 10. The first number designated the page, the second the position of the item within a page. For example, a keyword, page, and position combination could have been “marketing, 6, 4.”</p><p>We then performed each search by entering the keyword followed by a space and then “.ppt” (the common abbreviation for PowerPoint<sup>®</sup> slideshows). For example, if the keyword and page combination was “marketing, 6, 4,” the search term would have been “marketing.ppt,” which would yield a series of Google hits from which we would select the forth hit on the sixth page. In cases where the specified hit was not a slideshow, the next slideshow down would be selected, and if that entry was not a slideshow, we looked at the subsequent one, repeating this process until we found a slideshow. The minimum required length for slideshows was 15 slides, and the maximum allowed length was 100 slides; if slideshows contained fewer than 15 or more than 100 slides, we again selected the next slideshow in the list until we found a slideshow of suitable length.</p><p>In total, we selected 141 slideshows using this method, one of which was excluded because it clearly bridged categories, leaving us with a total of 140. We then classified each of these slideshows into one of five categories (in addition to Research, Education, Government, and Business, we defined a fifth category, Miscellaneous, for slideshows that did not fit clearly into the other four). Because we selected search terms that were targeted for a specific category, we were likely to find slideshows in that category – but our search method did not guarantee this result, and thus we double-checked the appropriate category after each slideshow was retrieved. Our search methods were designed to have a high probability of sampling from the different categories, and we had hoped to sample about the same number of slideshows from each category. However, our criteria were not perfect, and thus the numbers in each category are not precisely the same. Ultimately each category contained between 20 and 40 slideshows, specifically: Research: 27, Education: 38, Government: 20, Business: 32, and Miscellaneous: 23.</p><p>We note that we did not search for “.pptx” or “.key” or for other extensions. At the time the study was begun (2008), .pptx had not been available very long – and hence we worried about possible biases of sampling works from “early adopters.” In addition, the vast majority of slideshows are created with PowerPoint<sup>®</sup>, and we worried that Keynote<sup>®</sup> users might also represent a biased sample. Thus, we can only be confident that our results generalize to PowerPoint<sup>®</sup> slideshows <italic>per se</italic>. In addition, because we sampled from the web, our results can only generalize to other slideshows of the sort that are posted on the web (however, for our purposes, this is sufficient – if anything, these slideshows may be better than those not deemed worthy of being posted).</p></sec><sec><title>Scoring and coding</title><p>Four judges scored the 140 slideshows for violations of the 137 rules. Two of the judges were authors, and two were college-educated paid research assistants in the laboratory; at the time of coding, none of the judges knew whether or not they would subsequently do enough work on this project to be included as authors, and none were biased to find flaws (let alone flaws of specific types). Each rule was worded as a statement that, if it described a slide or slideshow as a whole, revealed a violation of that rule.</p><p>Two of the judges began by independently scoring 10 slideshows, and then comparing their scores. Any disparities were discussed, and in five cases the wording of rules was modified to be more specific. Following this initial training and calibration procedure, each judge independently scored all of the slideshows. The judges then discussed any disparities in classification or scoring, and reached a consensus.</p><p>The initial two judges interpreted each rule literally (“strictly”). For example, if the rule stated that no more than two lines should be included in a single bullet point, then a bullet point that had two lines and a single word in a third line would be classified as violating the rule. We adopted this procedure because we wanted to be sure that the rules could be easily interpreted. Nevertheless, we were concerned that we would inflate the number of violations by adhering rigidly to the rules; many of them were intended to be heuristics. For example, the prohibition against more than two lines per bullet point is based on the idea that no more than four concepts should be held in working memory at the same time, and in general two lines of text convey about four concepts.</p><p>Thus, following the initial scoring, two additional judges considered each of the violations identified by the first two judges. They decided, independently, whether each violation was important. In this case, “important” was defined as “likely to disrupt the comprehension or memory of the material.” If not, then they rejected that initial classification of a violation. By using both the strict and liberal scoring methods, we thereby defined a range in which we evaluated the slideshows. Table <xref ref-type="table" rid="T1">1</xref> shows the proportion of slideshows that violated each of the 137 rules, sorted by principle, scored separately for the strict and liberal criteria. Finally, two authors independently evaluated the rules and – using the summary provided in the introduction – indicated which principle applied to each rule. When they disagreed (for 15% of the rules), a discussion resolved the classification. In the process of this discussion, it became clear that 12% of the rules were “over-determined”: rather than following primarily from a single principle, they followed from two or more principles. Thus, we created a ninth level for the “principle” variable, which included all the rules for which more than one principle applied.</p></sec></sec><sec><title>Results</title><p>The dependent variable was whether or not a slideshow violated a principle. We took a “bad-apple-spoils-the-barrel” approach: if even a single slide contained material that violated a rule, the slideshow was scored as having violated the rule. And if a slideshow violated one or more rules within a specific principle, the score for that principle would be “1;” if none of the rules were violated for a principle, the score for it would be “0.” We present first the results from the strict, initial scoring, and after this the results from the more liberal scoring.</p><sec><title>Strict scoring</title><p>We began by assessing inter-rater reliability. The overall inter-rater agreement was 0.88, which indicates the proportion of times that the raters either both coded “1,” indicating a violation, or both coded “0,” indicating no violation. We then sought to answer three questions about the data.</p><p>First, we asked whether the violations of the principles differed for the different categories (see Table <xref ref-type="table" rid="T2">2</xref>). To answer this question, we conducted a repeated-measures ANOVA, with the nine levels of the Principles (the eight principles plus an “over-determined” class) as a within-participants variable and Category as a between-participants variable. The results showed that, in general, there were no overall differences in violations for the different categories, <italic>F</italic>(4, 135) = 2.24, <italic>p</italic> > 0.05, <inline-formula><mml:math id="M1"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>06</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula></p><table-wrap id="T2" position="float"><label>Table 2</label><caption><p><bold>Proportion of violations for each principle in each category for Study 1</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1">Category</th><th align="left" rowspan="1" colspan="1">Principle</th><th align="left" rowspan="1" colspan="1"><italic>N</italic></th><th colspan="2" align="center" rowspan="1">Strict scoring<hr></hr></th><th colspan="2" align="center" rowspan="1">Liberal scoring<hr></hr></th></tr><tr><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1">Mean</th><th align="left" rowspan="1" colspan="1">SD</th><th align="left" rowspan="1" colspan="1">Mean</th><th align="left" rowspan="1" colspan="1">SD</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Business</td><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.81</td><td align="left" rowspan="1" colspan="1">0.40</td><td align="left" rowspan="1" colspan="1">0.72</td><td align="left" rowspan="1" colspan="1">0.46</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.53</td><td align="left" rowspan="1" colspan="1">0.51</td><td align="left" rowspan="1" colspan="1">0.41</td><td align="left" rowspan="1" colspan="1">0.50</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">0.97</td><td align="left" rowspan="1" colspan="1">0.18</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Informative change</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.91</td><td align="left" rowspan="1" colspan="1">0.30</td><td align="left" rowspan="1" colspan="1">0.91</td><td align="left" rowspan="1" colspan="1">0.30</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.51</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.51</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.88</td><td align="left" rowspan="1" colspan="1">0.34</td><td align="left" rowspan="1" colspan="1">0.81</td><td align="left" rowspan="1" colspan="1">0.40</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.78</td><td align="left" rowspan="1" colspan="1">0.42</td><td align="left" rowspan="1" colspan="1">0.75</td><td align="left" rowspan="1" colspan="1">0.44</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">“Over-determined”</td><td align="left" rowspan="1" colspan="1">32</td><td align="left" rowspan="1" colspan="1">0.56</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.53</td><td align="left" rowspan="1" colspan="1">0.51</td></tr><tr><td align="left" rowspan="1" colspan="1">Research</td><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.74</td><td align="left" rowspan="1" colspan="1">0.45</td><td align="left" rowspan="1" colspan="1">0.67</td><td align="left" rowspan="1" colspan="1">0.48</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.15</td><td align="left" rowspan="1" colspan="1">0.36</td><td align="left" rowspan="1" colspan="1">0.11</td><td align="left" rowspan="1" colspan="1">0.32</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Informative change</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.85</td><td align="left" rowspan="1" colspan="1">0.36</td><td align="left" rowspan="1" colspan="1">0.74</td><td align="left" rowspan="1" colspan="1">0.45</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.26</td><td align="left" rowspan="1" colspan="1">0.45</td><td align="left" rowspan="1" colspan="1">0.22</td><td align="left" rowspan="1" colspan="1">0.42</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.74</td><td align="left" rowspan="1" colspan="1">0.45</td><td align="left" rowspan="1" colspan="1">0.74</td><td align="left" rowspan="1" colspan="1">0.45</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.70</td><td align="left" rowspan="1" colspan="1">0.47</td><td align="left" rowspan="1" colspan="1">0.67</td><td align="left" rowspan="1" colspan="1">0.48</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">“Over-determined”</td><td align="left" rowspan="1" colspan="1">27</td><td align="left" rowspan="1" colspan="1">0.41</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.37</td><td align="left" rowspan="1" colspan="1">0.49</td></tr><tr><td align="left" rowspan="1" colspan="1">Government</td><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.51</td><td align="left" rowspan="1" colspan="1">0.40</td><td align="left" rowspan="1" colspan="1">0.50</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.40</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.35</td><td align="left" rowspan="1" colspan="1">0.49</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">0.95</td><td align="left" rowspan="1" colspan="1">0.22</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Informative change</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.90</td><td align="left" rowspan="1" colspan="1">0.31</td><td align="left" rowspan="1" colspan="1">0.85</td><td align="left" rowspan="1" colspan="1">0.37</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.40</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.40</td><td align="left" rowspan="1" colspan="1">0.50</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.75</td><td align="left" rowspan="1" colspan="1">0.44</td><td align="left" rowspan="1" colspan="1">0.70</td><td align="left" rowspan="1" colspan="1">0.47</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.65</td><td align="left" rowspan="1" colspan="1">0.49</td><td align="left" rowspan="1" colspan="1">0.65</td><td align="left" rowspan="1" colspan="1">0.49</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">“Over-determined”</td><td align="left" rowspan="1" colspan="1">20</td><td align="left" rowspan="1" colspan="1">0.35</td><td align="left" rowspan="1" colspan="1">0.49</td><td align="left" rowspan="1" colspan="1">0.35</td><td align="left" rowspan="1" colspan="1">0.49</td></tr><tr><td align="left" rowspan="1" colspan="1">Education</td><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.74</td><td align="left" rowspan="1" colspan="1">0.45</td><td align="left" rowspan="1" colspan="1">0.61</td><td align="left" rowspan="1" colspan="1">0.50</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.26</td><td align="left" rowspan="1" colspan="1">0.45</td><td align="left" rowspan="1" colspan="1">0.26</td><td align="left" rowspan="1" colspan="1">0.45</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Informative change</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">0.89</td><td align="left" rowspan="1" colspan="1">0.31</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.34</td><td align="left" rowspan="1" colspan="1">0.48</td><td align="left" rowspan="1" colspan="1">0.34</td><td align="left" rowspan="1" colspan="1">0.48</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.76</td><td align="left" rowspan="1" colspan="1">0.43</td><td align="left" rowspan="1" colspan="1">0.76</td><td align="left" rowspan="1" colspan="1">0.43</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.76</td><td align="left" rowspan="1" colspan="1">0.43</td><td align="left" rowspan="1" colspan="1">0.71</td><td align="left" rowspan="1" colspan="1">0.46</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">“Over-determined”</td><td align="left" rowspan="1" colspan="1">38</td><td align="left" rowspan="1" colspan="1">0.32</td><td align="left" rowspan="1" colspan="1">0.47</td><td align="left" rowspan="1" colspan="1">0.29</td><td align="left" rowspan="1" colspan="1">0.46</td></tr><tr><td align="left" rowspan="1" colspan="1">Miscellaneous</td><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.61</td><td align="left" rowspan="1" colspan="1">0.50</td><td align="left" rowspan="1" colspan="1">0.43</td><td align="left" rowspan="1" colspan="1">0.51</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.22</td><td align="left" rowspan="1" colspan="1">0.42</td><td align="left" rowspan="1" colspan="1">0.22</td><td align="left" rowspan="1" colspan="1">0.42</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">0.96</td><td align="left" rowspan="1" colspan="1">0.21</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Informative change</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.96</td><td align="left" rowspan="1" colspan="1">0.21</td><td align="left" rowspan="1" colspan="1">0.87</td><td align="left" rowspan="1" colspan="1">0.34</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td><td align="left" rowspan="1" colspan="1">1.00</td><td align="left" rowspan="1" colspan="1">0.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.30</td><td align="left" rowspan="1" colspan="1">0.47</td><td align="left" rowspan="1" colspan="1">0.30</td><td align="left" rowspan="1" colspan="1">0.47</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.91</td><td align="left" rowspan="1" colspan="1">0.29</td><td align="left" rowspan="1" colspan="1">0.83</td><td align="left" rowspan="1" colspan="1">0.39</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.87</td><td align="left" rowspan="1" colspan="1">0.34</td><td align="left" rowspan="1" colspan="1">0.83</td><td align="left" rowspan="1" colspan="1">0.39</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">“Over-determined”</td><td align="left" rowspan="1" colspan="1">23</td><td align="left" rowspan="1" colspan="1">0.09</td><td align="left" rowspan="1" colspan="1">0.29</td><td align="left" rowspan="1" colspan="1">0.09</td><td align="left" rowspan="1" colspan="1">0.29</td></tr></tbody></table></table-wrap><p>Second, we asked whether some principles were violated more frequently than others. As illustrated in Figure <xref ref-type="fig" rid="F1">1</xref>, in the same two-way ANOVA just noted, we found that some principles were violated more often than others, <italic>F</italic>(8, 1080) = 87.94, <italic>p</italic> < 0.01, <inline-formula><mml:math id="M2"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>39</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> A Bonferroni-corrected <italic>post hoc</italic> test with an alpha of 0.05 revealed that out of the 36 possible comparisons, only the following eight were <italic>not</italic> significant: the difference between Appropriate Knowledge and Salience, Compatibility and Over-determined, Compatibility and Perceptual Organization, Discriminability and Limited Capacity, Informative Change and Relevance, Informative Change and Salience, Over-determined and Perceptual Organization, and Relevance and Salience. All other comparisons were significant.</p><fig id="F1" position="float"><label>Figure 1</label><caption><p><bold>The percentage of presentations that had violations, scored according to strict (dark bars) and liberal (light bars) criteria, in Study 1</bold>. A, appropriate knowledge; C, compatibility; D, discriminability; I, informative change; L, limited capacity; P, perceptual organization; R, relevance; S, salience; O, over-determined. Error bars illustrate the standard error of the mean.</p></caption><graphic xlink:href="fpsyg-03-00230-g001"></graphic></fig><p>In addition, the results of this analysis showed there was an interaction between Principles and Category, <italic>F</italic>(32, 1080) = 1.91, <italic>p</italic> < 0.05, <inline-formula><mml:math id="M3"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>054</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> Specifically, a Bonferroni-corrected <italic>post hoc</italic> procedure documented that the difference in violations between the principle of Appropriate Knowledge and the principle of Compatibility was significantly different between the categories of Research and Government, <italic>F</italic>(1, 45) = 10.19, <italic>p</italic> < 0.01, <inline-formula><mml:math id="M4"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>19</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula></p><p>In general, the following aspects of these results are worth noting: the principles of Discriminability and Limited Capacity were violated in every single slideshow, and the principle of Informative Change was violated in 93% of the slideshows. In contrast, the principle of Compatibility was violated in only 31% of the slideshows.</p><p>Third, we examined which rules were violated most often. In particular, we found that the following five rules were violated most often: (1) Bulleted items are not presented individually, growing the list from the top to the bottom (96% of the slideshows); (2) More than two lines are used per bulleted sentence (91%); (3) More than four bulleted items appear in a single list (91%); (4) Hierarchical organization of lists is not used, with no more than four items at each level (86%); (5) All uppercase, all italics, or all bold typefaces are used (81%). To examine this result more carefully, we examined whether the five worst rules were violated equally often for the different categories of slideshows. The results showed no interaction between Category and the violation of the five worst rules, <italic>F</italic>(16, 540) = 1.51, <italic>p</italic> > 0.05, <inline-formula><mml:math id="M5"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>04</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> See Table <xref ref-type="table" rid="T3">3</xref> for a more detailed breakdown of the worst rules per category.</p><table-wrap id="T3" position="float"><label>Table 3</label><caption><p><bold>Most frequently violated rules per category (strict scoring) for Study 1</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1">Category</th><th align="left" rowspan="1" colspan="1">Rule</th><th align="left" rowspan="1" colspan="1">Proportion violated</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Business</td><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">0.97</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.97</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.94</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">0.81</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">All uppercase, all italics, or all bold typefaces are used</td><td align="left" rowspan="1" colspan="1">0.78</td></tr><tr><td align="left" rowspan="1" colspan="1">Research</td><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">1.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">1.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.85</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.81</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Words are not large enough (i.e., greater than 20 point) to be easily read</td><td align="left" rowspan="1" colspan="1">0.70</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Visual or auditory characteristics change even when they do not signal a change in information</td><td align="left" rowspan="1" colspan="1">0.70</td></tr><tr><td align="left" rowspan="1" colspan="1">Government</td><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">0.95</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">All uppercase, all italics, or all bold typefaces are used</td><td align="left" rowspan="1" colspan="1">0.95</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">0.95</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.95</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.90</td></tr><tr><td align="left" rowspan="1" colspan="1">Education</td><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">0.89</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">0.89</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.89</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Visual or auditory characteristics change even when they do not signal a change in information</td><td align="left" rowspan="1" colspan="1">0.84</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">All uppercase, all italics, or all bold typefaces are used</td><td align="left" rowspan="1" colspan="1">0.82</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.82</td></tr><tr><td align="left" rowspan="1" colspan="1">Miscellaneous</td><td align="left" rowspan="1" colspan="1">Bulleted items are not presented individually, growing the list from the top to the bottom</td><td align="left" rowspan="1" colspan="1">1.00</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than two lines are used per bulleted sentence</td><td align="left" rowspan="1" colspan="1">0.96</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">More than four bulleted items appear in a single list</td><td align="left" rowspan="1" colspan="1">0.91</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">All uppercase, all italics, or all bold typefaces are used</td><td align="left" rowspan="1" colspan="1">0.87</td></tr><tr><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">Hierarchical organization of lists is not used, with no more than four items at each level</td><td align="left" rowspan="1" colspan="1">0.83</td></tr></tbody></table></table-wrap></sec><sec><title>Liberal scoring</title><p>We again assessed inter-rater reliability, now based on the judgments with the more liberal criteria. The proportion agreement between the two raters for all principles and scales combined was 0.98. The shift from strict to liberal criteria did not change the pattern of the results. Using the new criteria resulted in our eliminating only 4% of the violations that were identified using the strict scoring procedure.</p><p>Considering again our three questions about the data: first, as with the strict coding, we conducted a repeated-measures ANOVA, with the nine levels of the Principles, as a within-participant variable, and Category as a between-participant variable. The results showed that, in general, there were no differences in violations between the categories, <italic>F</italic>(4, 135) = 1.93, <italic>p</italic> > 0.1, <inline-formula><mml:math id="M6"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>05</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula></p><p>Second, we again asked whether some principles were violated more often than others. As illustrated in Figure <xref ref-type="fig" rid="F1">1</xref>, we found that some principles were violated more often than others, <italic>F</italic>(8, 1080) = 79.52, <italic>p</italic> < 0.05, <inline-formula><mml:math id="M7"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>37</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> A Bonferroni-corrected <italic>post hoc</italic> test with an alpha of 0.05 showed that out of the 36 possible comparisons, only the following eight were <italic>not</italic> significant: the difference between Appropriate Knowledge and Salience, Compatibility and Over-determined, Compatibility and Perceptual Organization, Discriminability and Limited Capacity, Informative Change and Relevance, Informative Change and Salience, Over-determined and Perceptual Organization, and between Relevance and Salience. All other comparisons were significant.</p><p>As before, the principles of Discriminability (98%), Limited Capacity (100%), and Informative Change (86%) each were violated in the vast majority of slideshows. Also as before, the principle of Compatibility was violated least-frequently, in “only” about a quarter of the slideshows (27%) in this case. The results of the analysis also showed that there was no interaction between Category and Principle, <italic>F</italic>(32, 1080) = 1.44, <italic>p</italic> > 0.05, <inline-formula><mml:math id="M8"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>04</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula></p><p>Third, we again examined which rules were violated most often. In this analysis, we found that the following five rules were violated most often: (1) Bulleted items are not presented individually, growing the list from the top to the bottom (94% of the slideshows); (2) More than four bulleted items appear in a single list (88%); (3) More than two lines are used per bulleted sentence (87%); (4) Hierarchical organization of lists is not used, with no more than four items at each level (84%); (5) Words are not large enough (i.e., greater than 20 point) to be easily seen (65%). The first four rules are the same as the first four identified in the previous analysis, but now the rule “All uppercase, all italics, or all bold typefaces are used” was replaced by another rule that also stemmed from the principle of Discriminability, namely the rule regarding size. To examine this result more carefully, we examined whether there was an interaction between the five worst rules and categories, and found such an interaction, <italic>F</italic>(16, 540) = 2.35, <italic>p</italic> < 0.05, <inline-formula><mml:math id="M9"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>07</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> Specifically, a Bonferroni-corrected <italic>post hoc</italic> procedure showed that the difference in violations between the rule “Words are not large enough (i.e., greater than 20 point) to be easily seen” and the rule “More than two lines are used per bulleted sentence” was significantly different between the categories of Business and Government, <italic>F</italic>(1, 50) = 10.99, <italic>p</italic> < 0.01, <inline-formula><mml:math id="M10"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>18</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula></p><p>Table <xref ref-type="table" rid="T2">2</xref> provides a summary of the violations per category per principle, using both the liberal and the strict scoring method.</p></sec></sec><sec><title>Conclusion</title><p>As is clear, by either the strict or liberal scoring criteria, many slideshows are flawed. Indeed, not a single slideshow was scored – according to either set of criteria – as having no flaws. Using the strict criteria, each slideshow violated on average 6.23 of the nine classes of principles, which shrank slightly to 5.88 using the liberal criteria. Considering just the eight individual principles, excluding the “over-determined” group of rules, the overall violation was 6.17 for the strict coding and 5.86 for the liberal coding. The three most-violated principles were Discriminability (because material was too similar to be easily distinguished), Limited Capacity (because too much information was presented), and Informative Change (because changes in how information was presented did not actually reflect changes in the information being conveyed). Even the least-frequently violated principle, Compatibility, was still violated in at least a quarter of the slideshows.</p><p>In addition, the fact that violations were comparable across the different categories is of interest. We examined the different categories partly in an effort to consider the possible influence of different topics. Although slideshows of different topics (or areas) may differ in their average complexity, the results do not suggest that violations of specific principles generally vary for different topics. That said, we did find that some specific rules were violated more frequently in some categories than others, which could reflect differences in the complexities of the topics.</p><p>In sum, the present results suggest that the psychological principles either are not obvious or are obvious but often ignored.</p></sec></sec><sec><title>Study 2</title><p>The analysis of PowerPoint<sup>®</sup> slideshows in Study 1 revealed that slideshows themselves typically are flawed in multiple ways. We wanted to know whether viewers are sensitive to presentation flaws more generally, and thus we conducted a survey asking participants to report approximately how many electronic slideshow presentations they had seen in the past year, and then to rate various aspects of those presentations in terms of their quality.</p><sec><title>Materials and methods</title><sec><title>Participants</title><p>We recruited 265 participants via ads on Craigslist and by word-of-mouth. In order to participate in the study, participants had to be at least 18 years old, speak fluent English, and “regularly” attend presentations that involved PowerPoint<sup>®</sup>, Keynote<sup>®</sup>, or other electronic slideshows. (In order to receive compensation – either a $5 gift certificate to <uri xlink:type="simple" xlink:href="http://Amazon.com">Amazon.com</uri> or a free book – they had to be US citizens or permanent residents and not employed by Harvard University.) All participants provided their informed consent and were tested in accordance with the ethical guidelines of the American Psychological Association; this research was approved by the Harvard University Committee on the Use of Human Subjects.</p><p>Forty-six of the participants did not meet the eligibility criteria of viewing at least one presentation per month during the previous year, and 14 skipped more than 20% of the questions. Therefore, analyses were conducted on data from the 205 eligible participants who completed at least 80% of the survey, including 112 females (mean age 32.5 ± 8.9) and 83 males (mean age 30.9 ± 8.9) and 10 who did not report their sex or age. According to self-report, about 7% of the participants were engineers, 10% were in finance, 13% were business managers or executives, and 14% were students; the remaining participants were distributed across a variety of different occupations.</p></sec><sec><title>Materials</title><p>The study was administered over the internet via a secured link (SSL) on <uri xlink:type="simple" xlink:href="http://Surveymonkey.com">Surveymonkey.com</uri>. The opening page of the survey consisted of a statement about the eligibility requirements and a description of the study content. Participants were assured that their data would be collected anonymously, and that they would be sent to another unrelated survey in order to provide the personal information (email address and physical address) we needed in order to send the gift certificate.</p><p>The survey began by asking approximately how many electronic slideshow presentations the participant viewed each year (up to “5 or more per week”). The main portion of the survey consisted of a series of questions about electronic slideshow presentations viewed in the last year. We asked two sorts of questions. First, we asked about the presence of specific problems; for example, we asked “Did any of the presentations fail to convey a meaningful message because there was no main point?” and “Were any of the presentations hard to follow because the slides contained too much material to absorb before the next slide was presented?” Participants were asked to indicate the proportion of presentations – if any – that had such problems, using the following scale: “None (of the presentations); Some; Half; Many; Virtually all.” Second, we asked questions about annoyance, querying the degree to which the problem, if present, bothered them; these responses used a four-point scale, labeled as follows: “Not at all; Somewhat; A fair amount; It was extremely annoying to have my time wasted in this fashion.” It could be argued that by focusing on problems, rather than framing the questions in a neutral fashion, we were biasing the participants to make negative responses – and this is a possible limitation of the survey. However, it is easier, grammatically and logistically, to ask people to notice problems than to notice things that were flawless. If a presentation is excellent the audience will be focused on the presenter’s message, not on the technicalities of the presentation.</p><p>The questions covered typical violations of seven of the eight principles. Because we were asking for overall impressions across all presentations recalled from the past year, rather than analyses of specific slides, we did not ask any questions about Perceptual Organization, fearing that participants would not be able to recall these types of details accurately. We included 12 questions about Relevance; 8 about Appropriate Knowledge; 4 about Salience; 5 about Discriminability; 3 about Compatibility; 2 about Informative Change; 7 about Limited Capacity; and one that did not clearly fit into any of these categories, “Were any of the speakers unprepared in that they didn’t know how to use their technology (software, computer, projector, etc.)?” We also included “free response” questions, asking participants to report any other common problems <italic>and</italic> any techniques that they thought contributed to especially effective presentations. The final portion of the survey asked for basic demographic information, including age, sex, and occupation (which they could provide explicitly or select from a checklist, or both).</p></sec></sec><sec><title>Results</title><p>The Appendix presents a table with all of the questions, the average ratings, and the percent responses in each category of the rating scale, for both prevalence and annoyance. The five items with the highest ratings (bold and underlined) and the five with the lowest (italics) for Prevalence and Annoyance are highlighted in gray. Note that one of the most prevalent violations (lack of a pointer) was one of the least annoying.</p><p>We coded the data in two ways. First, we converted the five “prevalence of violation” and four “annoyance” ratings into numerical values (0–4, and 0–3, respectively), and then for each participant we calculated the average rating for prevalence of violations and for associated annoyance level for each principle. Because the most common answers were “none” and “some,” we also coded the data using a binary coding system, coding any degree of violation [“some (presentations),” “half,” “many,” “virtually all”] as “1,” and “none” as “0,” and any degree of annoyance (“somewhat,” “a fair amount,” “it was extremely annoying to have my time wasted in this fashion”) as “1,” and “not at all” as “0,” and then again for each participant calculated an average score for both prevalence and annoyance for each of the principles. (The fact that these were the most common answers suggests that participants were not biased by our questions.)</p><p>As in Study 1, we asked whether some principles were violated more often than others, calculating one-way repeated-measures ANOVAs. For all ANOVAs Mauchly’s test indicated that the assumption of sphericity had been violated (chi-square > 163, <italic>p</italic> < 0.0005, epsilon > 0.75), therefore degrees of freedom were corrected using Huynh–Feldt estimates of sphericity. The results were similar regardless of the coding used (full scale or binary), revealing that some of the principles were violated more often than others, <italic>F</italic>(4.5, 927) = 10.80, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M11"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>05</mml:mn></mml:mrow></mml:math></inline-formula> for the full scale, and <italic>F</italic>(4.9, 999) = 8.00, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M12"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>04</mml:mn></mml:mrow></mml:math></inline-formula> for the binary scale. Overall, respondents reported that all seven principles were violated at least some of the time (see Table <xref ref-type="table" rid="T4">4</xref>). These reports ranged from a low of 63% of respondents reporting at least some presentations with violations of Compatibility to a high of 74% reporting at least some with violations of Salience.</p><table-wrap id="T4" position="float"><label>Table 4</label><caption><p><bold>Percentage of respondents reporting “at least some” prevalence of violations and associated annoyance, by principle, in Study 2</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1">Principle</th><th align="left" rowspan="1" colspan="1">Prevalence</th><th align="left" rowspan="1" colspan="1">Annoyance</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">72.5</td><td align="left" rowspan="1" colspan="1">65.2</td></tr><tr><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">62.5</td><td align="left" rowspan="1" colspan="1">60.5</td></tr><tr><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">65.4</td><td align="left" rowspan="1" colspan="1">65.0</td></tr><tr><td align="left" rowspan="1" colspan="1">Informative changes</td><td align="left" rowspan="1" colspan="1">67.3</td><td align="left" rowspan="1" colspan="1">58.6</td></tr><tr><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">70.1</td><td align="left" rowspan="1" colspan="1">65.0</td></tr><tr><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">68.4</td><td align="left" rowspan="1" colspan="1">64.6</td></tr><tr><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">73.5</td><td align="left" rowspan="1" colspan="1">62.2</td></tr></tbody></table></table-wrap><p>We also asked whether violations of some principles were more annoying than violations of other principles. (Data from 197 participants, rather than 205, were used in these analyses, because some of those who reported no violations skipped the “annoyance” questions.) One-way repeated-measures ANOVAs revealed that violations of some principles were more annoying than others <italic>F</italic>(4.9, 968) = 5.15, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M13"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>03</mml:mn></mml:mrow></mml:math></inline-formula> (full scale), and <italic>F</italic>(4.7, 928) = 3.05, <italic>p</italic> < 0.05, <inline-formula><mml:math id="M14"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>02</mml:mn></mml:mrow></mml:math></inline-formula> (binary); these results are also presented in Table <xref ref-type="table" rid="T4">4</xref>. The range was similar to the previous measure, but different principles anchored it: nearly 59% of respondents reported at least some annoyance with violations of Informative Change, whereas over 65% reported at least some annoyance with violations of Appropriate Knowledge.</p><p>Although we did document a significant difference in reported prevalence and annoyance for the violations of different principles, it was relatively small and the most interesting results may be in the responses to individual questions. The three most prevalent problems were also among the five most annoying: “Speakers read word-for-word from notes or from the slides themselves” (Salience), “The slides contained too much material to absorb before the next slide was presented” (Limited Capacity), and (most prevalent and most annoying), “The main point was obscured by lots of irrelevant detail” (Relevance).</p><p>Because different numbers of questions were asked for different principles, we conducted correlation analyses to discover whether the mean ratings were related to the number of questions per principle. No significant correlations were found between prevalence or annoyance ratings and the number of questions per principle, nor were prevalence and annoyance significantly correlated with each other.</p><p>We also invited the participants to make free responses about presentation problems that were not covered in the survey, and to indicate key aspects of especially good presentations. These free responses, in general, tended to reiterate topics covered in the survey, for instance, “When explaining complicated information, it’s important to use as many slides as necessary. Do not try and cram it all together.” Or, “a little humor goes a long way.” However, many respondents complained that speakers should be better prepared, by “rehearsing their lectures so they don’t look like it’s the first time they’re seeing these slides too,” and by “hav(ing) everything ready and working properly before (the audience) show(s) up” and coordinating carefully with any assistants who operate the equipment so that technical problems do not derail the entire talk. (About 61% of the respondents reported that at least some of the presentations were flawed because the presenter did not know how to use his/her equipment, and more than 50% of the respondents indicated that this was at least somewhat annoying.) In short, “Don’t waste my time! If you aren’t going to properly prepare just send me a memo with information instead!”</p></sec><sec><title>Conclusion</title><p>These results converge with those from the Study 1, namely that all seven principles we asked about (out of eight considered in the first study) are violated on a regular basis. Such violations bother audience members to some extent. And as with Study 1, violations of the Principle of Compatibility were least common, whereas one of the most annoying and prevalent violations fell under the principle of Limited Capacity (slides contained too much material to absorb before the next slide was presented). These results show that presentations not only have much room for improvement, as shown in Study 1, but also that audience members are sensitive to the general sorts of problems that plague presentations.</p><p>The findings from Study 1 and Study 2 thus raise an interesting question: why do at least some people feel that the psychological principles are so obvious that nobody would violate them? Is it possible that although people recognize overall problems – such as that a presentation is boring or unclear – they fail to recognize the precise problems with specific slides that produce such easily noticed flaws? We investigate this possibility in the following study.</p></sec></sec><sec><title>Study 3</title><p>The present study was designed to answer two questions: first, we wanted to know whether people can identify graphical displays in individual slides in which the different principles have been violated. Second, if they can identify such defective displays, we wanted to know whether participants are aware of <italic>why</italic> a display is defective.</p><p>The previous two studies were surveys. Such studies have the advantage of tapping into phenomena that occur in the “real world,” but the disadvantage of not having rigorous control over the relevant variables. Study 3 relies on an experiment. We selected a subset of the visual display design rules from Study 1, and prepared two illustrations for each one: one illustration included a violation of the rule and one did not (many of these pairs were taken directly from the “Do/Don’t” illustrations shown in Kosslyn, <xref ref-type="bibr" rid="B25">2007</xref>). Prior to presenting each pair of slides, we presented a question that guided participants to focus on a specific aspect of the slide. For example, for one slide pair we asked, “Which slide does a better job showing how closely two variables are related?” Participants were asked to indicate which slide was better, according to that specific criterion, and then to explain why they made their choice.</p><sec><title>Materials and methods</title><sec><title>Participants</title><p>A total of 53 participants took part in Study 3; however, five of these participants did not follow the instructions, so their data were not included in the analyses. The analyzed data came from 21 males with a mean age of 29.3 (SD = 11.1) and 27 females with a mean age of 23.6 (SD = 5.3). Participants were recruited through a Harvard University study pool, which was open to Harvard affiliates and members of the Cambridge/Boston community. None of the participants had prior knowledge of the presented material. All participants provided their informed consent and were tested in accordance with the ethical guidelines of the American Psychological Association; this research was approved by the Harvard University Committee on the Use of Human Subjects.</p></sec><sec><title>Materials</title><p>Forty-one “Do/Don’t” pairs of illustrations from Kosslyn (<xref ref-type="bibr" rid="B25">2007</xref>) formed the core of our stimulus pairs; we removed the “Do/Don’t” labels from these illustrations. We created 23 additional pairs of illustrations (some from Kosslyn, <xref ref-type="bibr" rid="B24">2006</xref>), to ensure that we had adequate examples for each of the eight principles. Four pairs were eventually eliminated because of problems with the quality of the illustrations, leaving us with six pairs of illustrations for Appropriate Knowledge, seven each for Perceptual Grouping and Salience, and eight for each of the other principles. We randomized the presentation order across principles, with the constraint that two stimuli in a row could not probe the same principle. We then randomly put the version with the violation on the left for half of the slides, and on the right for the other half. No more than five slides in a row had the violation on the same side of the pair. We next created another set of stimuli by reversing the left and right panels for each pair (and thus, for each pair the violation appeared on the left in one version, and on the right in the other). Finally, we created two orders, forward and reverse, for each of the two stimulus sets, which resulted in a set of four stimulus sequences.</p><p>In addition, we prepared a question for each pair of illustrations. Each question asked participants to choose a panel (left or right) of the pair according to a particular criterion, such as: “Which slide does a better job of using textures?”</p><p>We tested groups of 3–10 participants together. Participants met in a computer lab, and each was seated at his or her own computer (Apple iMacs, with 20″ screens). We administered each of the four sets of stimuli equally often, and thus counterbalanced for presentation order (left/right and forward/reverse).</p><p>On each trial, participants first viewed the question and then the accompanying pair of illustrations. Participants indicated each choice by typing their answer (“l” or “r”) in a column in a spreadsheet. After choosing, the participants typed a brief explanation of their choice. The participants viewed the questions and pairs of stimuli at their own pace. We include examples of participant responses below, in addition to Figure <xref ref-type="fig" rid="F2">2</xref>, which illustrates a stimulus pair.</p><fig id="F2" position="float"><label>Figure 2</label><caption><p><bold>Example of a slide pair viewed by participants in Study 3 (illustrating the principle of Compatibility)</bold>. Prior to seeing this pair, they received the question, “Which slide does a better job of using a background pattern?” The correct answer for slide choice is the panel on the right. An example of a correct explanation is, “the topic of the chart is global warming, which is better represented with a sun than with clouds;” an example of an incorrect explanation is, “the right has more of an apocalyptic feel.”</p></caption><graphic xlink:href="fpsyg-03-00230-g002"></graphic></fig><p>For instance, we asked the question, “which slide does a better job using the key (the labeled squares at the top)?” The same bar graph was used in both slides. The better slide ordered the patches (squares) in the key to correspond with the order of the bars on the graph (which had the darkest bar in the middle); the flawed slide ordered the patches from lightest to darkest. A correct explanation for selecting the correct slide (matching order) was, “the key is laid out in the same sequence as the bars on the graph.” An incorrect explanation for selecting the correct slide (matching order) was, “the patterns in the key are arranged more dynamically with better contrast.” A completely incorrect response was to choose the non-matching order “because the key feels more orderly, the lightest shade on top and the darkest shade on the bottom.”</p><p>On another trial we asked, “Which slide uses the background pattern more effectively?” The two slides contained identical captioned and labeled line graphs with home prices over time; both had a photo of a house in the background. In the flawed slide, the background photo was inappropriately pronounced, making the graph difficult to read. The better slide used the same photo, but as a de-saturated watermark; it was visually interesting without in any way obscuring the graph. A correct explanation for selecting the correct slide (graph more salient than background) was, “[the background] is subtle but nice and doesn’t overshadow the graph.” An incorrect explanation for the correct slide (graph more salient) was, “it is more explanatory.” A completely incorrect response was to choose the slide with the too-salient background because, “you can see all of the beautiful scenery behind the home as well as the attractiveness of its surroundings.”</p></sec></sec><sec><title>Results</title><p>Because we asked the participants not only to choose one member from each pair, but also to explain their choice, we computed two types of scores: the error rate for <italic>slide choice</italic> and for <italic>explanation</italic>. Two judges (one of whom is an author of this article, and the other of whom was a paid research assistant in the lab) independently scored the explanations participants provided for each of their choices. These judges determined whether the participant had in fact identified the key feature that distinguished the versions of the illustration. We used “liberal” criteria, giving the participants the benefit of the doubt if there were any questions about their accuracy. Inter-rater agreement for the two judges was substantial, at 0.90 (ranging from a low of 0.87 for the principle of Perceptual Grouping to a high of 0.93 for the principle of Relevance). When the judges disagreed in their initial scoring, they discussed the item in question and came to a consensus agreement. Table <xref ref-type="table" rid="T5">5</xref> and Figure <xref ref-type="fig" rid="F3">3</xref> present the results for slide choice and explanation organized according to principle.</p><table-wrap id="T5" position="float"><label>Table 5</label><caption><p><bold>Percentage of incorrect slide choices and incorrect explanations of correct choices, by principle, in Study 3</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1">Principle</th><th align="left" rowspan="1" colspan="1">Incorrect choices</th><th align="left" rowspan="1" colspan="1">Incorrect explanations following correct choice</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">6.6</td><td align="left" rowspan="1" colspan="1">10.8</td></tr><tr><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">28.9</td><td align="left" rowspan="1" colspan="1">19.3</td></tr><tr><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">13.3</td><td align="left" rowspan="1" colspan="1">20.8</td></tr><tr><td align="left" rowspan="1" colspan="1">Informative changes</td><td align="left" rowspan="1" colspan="1">18.2</td><td align="left" rowspan="1" colspan="1">21.8</td></tr><tr><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">17.2</td><td align="left" rowspan="1" colspan="1">17.9</td></tr><tr><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">25.0</td><td align="left" rowspan="1" colspan="1">15.3</td></tr><tr><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">26.0</td><td align="left" rowspan="1" colspan="1">14.6</td></tr><tr><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">28.9</td><td align="left" rowspan="1" colspan="1">14.3</td></tr></tbody></table></table-wrap><fig id="F3" position="float"><label>Figure 3</label><caption><p><bold>The percentage of trials on which participants made errors in choosing the correct slide (dark bars), and explaining their choice (light bars) given that they chose the correct slide, in Study 3</bold>. A, appropriate knowledge; C, compatibility; D, discriminability; I, informative change; L, limited capacity; P, perceptual organization; R, relevance; S, salience. Error bars illustrate the standard error of the mean.</p></caption><graphic xlink:href="fpsyg-03-00230-g003"></graphic></fig><p>The mean error rate for slide choice was 20.5% (with chance performance at 50% errors). A repeated-measures ANOVA revealed that the error rates for slide choice varied for the different principles, <italic>F</italic>(7, 329) = 22.60, <italic>p</italic> < 0.0005, <inline-formula><mml:math id="M15"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>325</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> As is evident in Table <xref ref-type="table" rid="T5">5</xref> and Figure <xref ref-type="fig" rid="F3">3</xref>, the participants were most accurate for the principle of Appropriate Knowledge and least accurate for the principles of Compatibility and Salience. Why did these participants have such a problem with Compatibility, when this principle is the least violated in the other studies? A close look at the results indicated that the present finding is driven by rules for very specific types of displays that may not be used frequently (and thus may not have been encountered in the presentations sampled in Study 1 or queried in Study 2).</p><p>The five stimulus pairs that induced the most errors in slide choice illustrated the following rules: explode a max of 25% of wedges in a pie (Salience); use a line graph to display interactions over two entities on the <italic>X</italic> axis (Compatibility); use different colors only for emphasis or to specify different classes of information (Informative Change); use a scatterplot to convey an overall impression of the relationship between two variables (Compatibility); use one color for titles and another, less salient one for text (Salience).</p><p>Although the participants chose the correct alternative on about 80% of the trials, on fully one sixth of these trials, they could not articulate why they made these choices; that is, 16.8% of the explanations for correct choices were incorrect. Thus, for only about two thirds of the total trials did they choose correctly on the basis of knowledge. A repeated-measures ANOVA showed that the participants made more errors for some principles than others when explaining their choices, <italic>F</italic>(7, 279) = 3.47, <italic>p</italic> = 0.003, <inline-formula><mml:math id="M16"><mml:mrow><mml:msubsup><mml:mrow><mml:mi>η</mml:mi></mml:mrow><mml:mrow><mml:mi>p</mml:mi></mml:mrow><mml:mrow><mml:mn>2</mml:mn></mml:mrow></mml:msubsup><mml:mo class="MathClass-rel">=</mml:mo><mml:mn>0</mml:mn><mml:mo class="MathClass-punc">.</mml:mo><mml:mn>069</mml:mn><mml:mo class="MathClass-punc">.</mml:mo></mml:mrow></mml:math></inline-formula> (Mauchly’s test indicated that the assumption of sphericity had been violated: chi-square = 59, <italic>p</italic> < 0.0005, epsilon = 0.847, therefore degrees of freedom were corrected using Huynh–Feldt estimates of sphericity.) The participants made relatively more errors when explaining violations of the principles of Discriminability and Informative Change, and were relatively accurate when explaining their choices for the principle of Appropriate Knowledge. Participants had the most difficulty explaining their (correct) choices for displays that illustrated the following rules: avoid using red and blue in adjacent regions (Discriminability); use hierarchical labeling (Limited Capacity); ensure that shapes of meaningful regions are easily identifiable (Discriminability); use one color for titles and another, less salient one for text (Salience); graph different types of data in a single display only if they are highly related and must be compared (Relevance).</p><p>Finally, Table <xref ref-type="table" rid="T6">6</xref> presents the percentage of participants who made <italic>at least one error</italic> in slide choice per principle, and (given they made the <italic>correct</italic> slide choice) <italic>at least one error</italic> in explanation per principle; note that there are no principles for which there are no errors.</p><table-wrap id="T6" position="float"><label>Table 6</label><caption><p><bold>Percentage of participants who made <italic>at least one erro</italic><italic>r</italic> in slide choice per principle, and percentage of participants who (given they made the correct slide choice) made <italic>at least one error</italic> in explanation per principle, in Study 3</bold>.</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1">Principle</th><th align="left" rowspan="1" colspan="1">Slide choice</th><th align="left" rowspan="1" colspan="1">Explanation</th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Appropriate knowledge</td><td align="left" rowspan="1" colspan="1">33.3</td><td align="left" rowspan="1" colspan="1">39.6</td></tr><tr><td align="left" rowspan="1" colspan="1">Compatibility</td><td align="left" rowspan="1" colspan="1">97.9</td><td align="left" rowspan="1" colspan="1">54.2</td></tr><tr><td align="left" rowspan="1" colspan="1">Discriminability</td><td align="left" rowspan="1" colspan="1">62.5</td><td align="left" rowspan="1" colspan="1">85.4</td></tr><tr><td align="left" rowspan="1" colspan="1">Informative changes</td><td align="left" rowspan="1" colspan="1">85.4</td><td align="left" rowspan="1" colspan="1">60.4</td></tr><tr><td align="left" rowspan="1" colspan="1">Limited capacity</td><td align="left" rowspan="1" colspan="1">75.0</td><td align="left" rowspan="1" colspan="1">62.5</td></tr><tr><td align="left" rowspan="1" colspan="1">Perceptual organization</td><td align="left" rowspan="1" colspan="1">81.3</td><td align="left" rowspan="1" colspan="1">45.8</td></tr><tr><td align="left" rowspan="1" colspan="1">Relevance</td><td align="left" rowspan="1" colspan="1">87.5</td><td align="left" rowspan="1" colspan="1">45.8</td></tr><tr><td align="left" rowspan="1" colspan="1">Salience</td><td align="left" rowspan="1" colspan="1">89.6</td><td align="left" rowspan="1" colspan="1">47.9</td></tr></tbody></table></table-wrap></sec><sec><title>Conclusion</title><p>In short, even when guided to pay attention to specific features of slides, observers do not always understand when the psychological principles have been violated, and even when they do realize that there is a problem, they often are not aware of the nature of this problem. Given the way information processing works in the brain, however, one could argue that even if viewers do not notice a violation, they could still be affected by it. Indeed, we would predict this to be the case, and it is well worth investigating this hypothesis in future research.</p></sec></sec><sec sec-type="discussion"><title>Discussion</title><p>In the three studies reported here, we analyzed how well PowerPoint<sup>®</sup> slideshows, slides, and presentations respect principles of human perception, memory, and comprehension. Specifically, we hypothesized and found that the psychological principles are often violated in PowerPoint<sup>®</sup> slideshows across different fields (Study 1), that some types of presentation flaws are noticeable and annoying to audience members (Study 2), and that observers have difficulty identifying many violations in graphical displays in individual slides (Study 3). These studies converge in painting the following picture: PowerPoint<sup>®</sup> presentations are commonly flawed; some types of flaws are more common than others; flaws are not isolated to one domain or context; and, although some types of flaws annoy the audience, flaws at the level of slide design are not always obvious to an untrained observer (but, we would argue, such flaws may negatively impact information processing). Specifically, in Study 1 we found that PowerPoint<sup>®</sup> slideshows on average violate approximately six of the eight principles we considered. Moreover, this is true for slideshows produced in different fields. The principles of Discriminability, Limited Capacity, and Informative Change were most frequently violated. In Study 2 we found that respondents commonly reported having noticed and been annoyed by problems in presentations in general, and these problems arose from violations of most of the principles we considered. Finally, in Study 3 we showed that observers often did not recognize when a specific rule was violated in a given slide, and that even when they did select the slide without the violation, they often could not explain their choice.</p><p>But can we be confident that violations of our principles really constitute “flaws” in a presentation? As noted in the Introduction, our principles grow out of a straightforward task analysis, which in turn led us to consider basic findings in the scientific literature on human perception, memory, and comprehension. Because of this connection, the principles have face validity: if a presentation violates a principle, it taxes human information processing. And because the presentation taxes information processing, the audience members will have difficulty perceiving, remembering, or comprehending it – and hence, it can reasonably be said to be flawed.</p><p>Nevertheless, future studies should explicitly examine the effects of violating specific principles on comprehension and subsequent memory. Such studies should also assess the degree to which audience members enjoy presentations with specific flaws, and their ability to think critically about such presentations (both while they are underway and subsequently). In addition, future studies should address whether specific types of violations have the same effects for slideshows that are presented versus for slideshows that are posted or distributed (and hence meant to be understood independently of a presenter).</p><p>By the same token, one could ask whether the rules “measure what they are supposed to measure.” To be clear: the rules do not “measure” anything; we are not presenting a new psychometric instrument. The rules are guidelines that should be obeyed to avoid including characteristics that will impair information processing. In the current research, we devised rules to capture key characteristics that would make presentations difficult to perceive, remember, or comprehend. As such, the “psychometric properties” associated with such a checklist are inherently different from a formal psychological test. For example, the fact that a specific rule is violated by every single presentation is, in the current study, very informative – whereas this is not true for an item in a conventional psychological test.</p><p>It is worth noting that respecting the principles and rules we have described will not produce “optimal” presentations. Instead, respecting the principles and rules will ensure that presentations are not flawed in these specific ways. Simply avoiding the glitches that we document will not ensure that the presentation is good, but including such glitches will make a presentation bad (or at least not as good as it could be). Given that the principles and rules are rooted in the empirical literature, avoiding violating them will improve a presentation. (However, we also must note that this claim is based on face validity, and must be explicitly tested in subsequent work).</p><p>We also must point out that in two of the three studies reported here, we treated the slides independently of the presenter. One could argue that at least some of the potential “flaws” may not be as severe when the slides are discussed by a skilled presenter. For instance, a talented presenter may be able to compensate for a particular flaw (e.g., by reading aloud text that is so small as to be all-but-invisible for the audience members). However, all else being equal, there is no advantage – and considerable disadvantage – to showing flawed slides. Although some flaws may be compensated for by talented presenters, it would be better if they did not have to compensate in the first place.</p><p>In short, we have documented that there is plenty of room for improvement in the design of electronic slideshow presentations. The psychological principles apparently are not entirely obvious, nor do people always respect them instinctively. Perhaps the best way to avoid such flaws is to prepare a slideshow draft, review each slide according to the psychological principles and rules we describe here, and then correct flaws as they are detected. Presenters may also want to review the Appendix in order to avoid common presentation mistakes, particularly those that the audience reports as highly annoying. It is worth noting that presentation techniques designed to compensate for poorly designed slides (such as reading aloud slides with miniscule text), may sometimes backfire – causing the audience to lose interest and tune out. Making your audience do unnecessary work is not a recipe for a successful presentation.</p></sec><sec><title>Conflict of Interest Statement</title><p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p></sec></body><back><ack><p>The authors thank William Thompson, Katie Lewis, Pooja Patnaik, and Roland Fuoco for technical assistance, and Sam Moulton for helpful comments. We also thank Ellen Winner, whose comment on a review on the Amazon website for Kosslyn (<xref ref-type="bibr" rid="B25">2007</xref>) provided the inspiration for Study 3. This research was supported, in part, by NSF Grant REC0411725 (the opinions and viewpoints expressed in this article are those of the authors, and do not necessarily reflect the opinions and viewpoints of the National Science Foundation).</p></ack><app-group><app id="A1"><title>Appendix</title><p>Study 2 survey questions, average ratings, and percent responses in each ratings category for <italic>Prevalence</italic> (“What proportion of presentations?” Ratings categories: 0 = “none”; 1 = “some”; 2 = “half”; 3 = “many”; 4 = “virtually all”) and <italic>Annoyance</italic> (“Did this bother you?” Ratings categories: 0 = “not at all”; 1 = “somewhat”; 2 = “a fair amount”; 3 = “it was extremely annoying…”).</p><p>The five items with the highest ratings (bold and underlined) and the five with the lowest (italics) for Prevalence and Annoyance are highlighted in gray.</p><table-wrap id="T7" position="anchor"><table frame="hsides" rules="groups"><thead><tr><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1">Average rating</th><th colspan="5" align="center" rowspan="1">Prevalence<hr></hr></th><th align="left" rowspan="1" colspan="1">Average rating</th><th colspan="4" align="center" rowspan="1">Annoyance<hr></hr></th></tr><tr><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1"></th><th align="left" rowspan="1" colspan="1"></th><th colspan="5" align="center" rowspan="1">Percent responses in each rating category (0–4 scale)</th><th align="left" rowspan="1" colspan="1"></th><th colspan="4" align="center" rowspan="1">Percent responses in each rating category (0–3)</th></tr><tr><th align="left" colspan="13" rowspan="1"><hr></hr></th></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Principle</td><td align="left" rowspan="1" colspan="1">Item</td><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">0</td><td align="left" rowspan="1" colspan="1">1</td><td align="left" rowspan="1" colspan="1">2</td><td align="left" rowspan="1" colspan="1">3</td><td align="left" rowspan="1" colspan="1">4</td><td align="left" rowspan="1" colspan="1"></td><td align="left" rowspan="1" colspan="1">0</td><td align="left" rowspan="1" colspan="1">1</td><td align="left" rowspan="1" colspan="1">2</td><td align="left" rowspan="1" colspan="1">3</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>DID ANY OF THE PRESENTATIONS FAIL TO CONVEY A MEANINGFUL MESSAGE BECAUSE</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">There was no main point</td><td align="left" rowspan="1" colspan="1">1.039</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">48.3</td><td align="left" rowspan="1" colspan="1">14.6</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">2.4</td><td align="left" rowspan="1" colspan="1">1.170</td><td align="left" rowspan="1" colspan="1">32.7</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">17.6</td><td align="left" rowspan="1" colspan="1">15.6</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">The main point was obscured by lots of irrelevant detail</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.363</underline></bold></td><td align="left" rowspan="1" colspan="1">14.6</td><td align="left" rowspan="1" colspan="1">52.2</td><td align="left" rowspan="1" colspan="1">19.0</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">4.4</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.340</underline></bold></td><td align="left" rowspan="1" colspan="1">20.0</td><td align="left" rowspan="1" colspan="1">37.1</td><td align="left" rowspan="1" colspan="1">25.4</td><td align="left" rowspan="1" colspan="1">13.7</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Not enough information was provided to support the main point</td><td align="left" rowspan="1" colspan="1">1.201</td><td align="left" rowspan="1" colspan="1">21.5</td><td align="left" rowspan="1" colspan="1">50.2</td><td align="left" rowspan="1" colspan="1">15.6</td><td align="left" rowspan="1" colspan="1">10.7</td><td align="left" rowspan="1" colspan="1">1.5</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.274</underline></bold></td><td align="left" rowspan="1" colspan="1">22.9</td><td align="left" rowspan="1" colspan="1">36.1</td><td align="left" rowspan="1" colspan="1">24.9</td><td align="left" rowspan="1" colspan="1">12.2</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>DID THE SPEAKER EVER FAIL TO CONNECT WITH THE AUDIENCE BECAUSE HE OR SHE</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Was talking about a topic that the audience did not care about</td><td align="left" rowspan="1" colspan="1">1.146</td><td align="left" rowspan="1" colspan="1">23.4</td><td align="left" rowspan="1" colspan="1">52.7</td><td align="left" rowspan="1" colspan="1">12.2</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">2.4</td><td align="left" rowspan="1" colspan="1">1.162</td><td align="left" rowspan="1" colspan="1">29.8</td><td align="left" rowspan="1" colspan="1">36.6</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">15.1</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Did not explain the importance of the topic</td><td align="left" rowspan="1" colspan="1">1.063</td><td align="left" rowspan="1" colspan="1">29.8</td><td align="left" rowspan="1" colspan="1">45.4</td><td align="left" rowspan="1" colspan="1">14.6</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">1.0</td><td align="left" rowspan="1" colspan="1">1.046</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">42.4</td><td align="left" rowspan="1" colspan="1">20.0</td><td align="left" rowspan="1" colspan="1">5.9</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Assumed the audience knew more than it did</td><td align="left" rowspan="1" colspan="1">1.205</td><td align="left" rowspan="1" colspan="1">26.3</td><td align="left" rowspan="1" colspan="1">43.9</td><td align="left" rowspan="1" colspan="1">15.6</td><td align="left" rowspan="1" colspan="1">11.2</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">1.015</td><td align="left" rowspan="1" colspan="1">34.6</td><td align="left" rowspan="1" colspan="1">34.1</td><td align="left" rowspan="1" colspan="1">20.5</td><td align="left" rowspan="1" colspan="1">7.8</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Assumed the audience knew less than it did</td><td align="left" rowspan="1" colspan="1">1.166</td><td align="left" rowspan="1" colspan="1">25.9</td><td align="left" rowspan="1" colspan="1">48.3</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">9.8</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">1.096</td><td align="left" rowspan="1" colspan="1">32.2</td><td align="left" rowspan="1" colspan="1">31.7</td><td align="left" rowspan="1" colspan="1">23.9</td><td align="left" rowspan="1" colspan="1">8.8</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Used too much jargon</td><td align="left" rowspan="1" colspan="1">1.249</td><td align="left" rowspan="1" colspan="1">22.9</td><td align="left" rowspan="1" colspan="1">48.3</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">13.2</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">1.060</td><td align="left" rowspan="1" colspan="1">28.3</td><td align="left" rowspan="1" colspan="1">40.0</td><td align="left" rowspan="1" colspan="1">24.4</td><td align="left" rowspan="1" colspan="1">4.9</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Went through the presentation too fast</td><td align="left" rowspan="1" colspan="1">0.902</td><td align="left" rowspan="1" colspan="1">38.0</td><td align="left" rowspan="1" colspan="1">44.9</td><td align="left" rowspan="1" colspan="1">7.8</td><td align="left" rowspan="1" colspan="1">7.3</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">0.864</td><td align="left" rowspan="1" colspan="1">42.9</td><td align="left" rowspan="1" colspan="1">29.3</td><td align="left" rowspan="1" colspan="1">20.0</td><td align="left" rowspan="1" colspan="1">4.9</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Went through the presentation too slowly</td><td align="left" rowspan="1" colspan="1">1.265</td><td align="left" rowspan="1" colspan="1">22.4</td><td align="left" rowspan="1" colspan="1">46.3</td><td align="left" rowspan="1" colspan="1">16.1</td><td align="left" rowspan="1" colspan="1">11.2</td><td align="left" rowspan="1" colspan="1">3.4</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.262</underline></bold></td><td align="left" rowspan="1" colspan="1">24.9</td><td align="left" rowspan="1" colspan="1">31.7</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">11.2</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>WERE ANY OF THE PRESENTATIONS HARD TO FOLLOW BECAUSE THE SPEAKER</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Did not follow a clear narrative structure (with a beginning, middle, and end)</td><td align="left" rowspan="1" colspan="1">0.985</td><td align="left" rowspan="1" colspan="1">29.8</td><td align="left" rowspan="1" colspan="1">52.7</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">7.8</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">0.865</td><td align="left" rowspan="1" colspan="1">32.2</td><td align="left" rowspan="1" colspan="1">45.4</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">3.4</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Did not provide an overview of the talk in the introduction</td><td align="left" rowspan="1" colspan="1">0.995</td><td align="left" rowspan="1" colspan="1">33.2</td><td align="left" rowspan="1" colspan="1">46.8</td><td align="left" rowspan="1" colspan="1">8.8</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">2.4</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.788</italic></td><td align="left" rowspan="1" colspan="1">41.0</td><td align="left" rowspan="1" colspan="1">35.1</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">2.9</td></tr><tr><td align="left" rowspan="1" colspan="1">S</td><td align="left" rowspan="1" colspan="1">Did not use his/her slides to provide direction to the talk</td><td align="left" rowspan="1" colspan="1">1.064</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">48.8</td><td align="left" rowspan="1" colspan="1">12.2</td><td align="left" rowspan="1" colspan="1">6.3</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">0.883</td><td align="left" rowspan="1" colspan="1">35.6</td><td align="left" rowspan="1" colspan="1">40.0</td><td align="left" rowspan="1" colspan="1">15.6</td><td align="left" rowspan="1" colspan="1">4.4</td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Did not use his/her slides to illustrate complex or critical information</td><td align="left" rowspan="1" colspan="1">1.044</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">50.2</td><td align="left" rowspan="1" colspan="1">10.7</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">0.953</td><td align="left" rowspan="1" colspan="1">32.2</td><td align="left" rowspan="1" colspan="1">39.5</td><td align="left" rowspan="1" colspan="1">17.1</td><td align="left" rowspan="1" colspan="1">5.4</td></tr><tr><td align="left" rowspan="1" colspan="1">S</td><td align="left" rowspan="1" colspan="1">Did not use a pointer or otherwise direct audience attention to important details on the slides</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.279</underline></bold></td><td align="left" rowspan="1" colspan="1">28.3</td><td align="left" rowspan="1" colspan="1">43.9</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">9.8</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.680</italic></td><td align="left" rowspan="1" colspan="1">48.3</td><td align="left" rowspan="1" colspan="1">32.7</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">4.4</td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Did not provide a summary of his/her key points at the end</td><td align="left" rowspan="1" colspan="1">1.143</td><td align="left" rowspan="1" colspan="1">30.2</td><td align="left" rowspan="1" colspan="1">41.5</td><td align="left" rowspan="1" colspan="1">14.6</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">4.4</td><td align="left" rowspan="1" colspan="1">0.969</td><td align="left" rowspan="1" colspan="1">36.1</td><td align="left" rowspan="1" colspan="1">30.7</td><td align="left" rowspan="1" colspan="1">20.5</td><td align="left" rowspan="1" colspan="1">6.3</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Did not have a clear conclusion</td><td align="left" rowspan="1" colspan="1">1.114</td><td align="left" rowspan="1" colspan="1">27.8</td><td align="left" rowspan="1" colspan="1">46.3</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">7.3</td><td align="left" rowspan="1" colspan="1">3.9</td><td align="left" rowspan="1" colspan="1">1.089</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">19.0</td><td align="left" rowspan="1" colspan="1">8.3</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>WERE ANY OF THE PRESENTATIONS HARD TO FOLLOW BECAUSE THE SLIDES</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Contained too much material to absorb before the next slide was presented</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.356</underline></bold></td><td align="left" rowspan="1" colspan="1">18.0</td><td align="left" rowspan="1" colspan="1">45.4</td><td align="left" rowspan="1" colspan="1">21.5</td><td align="left" rowspan="1" colspan="1">13.2</td><td align="left" rowspan="1" colspan="1">2.0</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.225</underline></bold></td><td align="left" rowspan="1" colspan="1">22.9</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">8.8</td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Had too many bullets</td><td align="left" rowspan="1" colspan="1">1.039</td><td align="left" rowspan="1" colspan="1">33.2</td><td align="left" rowspan="1" colspan="1">44.4</td><td align="left" rowspan="1" colspan="1">9.8</td><td align="left" rowspan="1" colspan="1">10.7</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">0.908</td><td align="left" rowspan="1" colspan="1">35.1</td><td align="left" rowspan="1" colspan="1">40.0</td><td align="left" rowspan="1" colspan="1">14.6</td><td align="left" rowspan="1" colspan="1">5.9</td></tr><tr><td align="left" rowspan="1" colspan="1">D</td><td align="left" rowspan="1" colspan="1">Had text or graphics that were too small to see clearly</td><td align="left" rowspan="1" colspan="1">1.171</td><td align="left" rowspan="1" colspan="1">24.4</td><td align="left" rowspan="1" colspan="1">50.7</td><td align="left" rowspan="1" colspan="1">10.7</td><td align="left" rowspan="1" colspan="1">11.7</td><td align="left" rowspan="1" colspan="1">2.4</td><td align="left" rowspan="1" colspan="1">1.195</td><td align="left" rowspan="1" colspan="1">25.9</td><td align="left" rowspan="1" colspan="1">34.6</td><td align="left" rowspan="1" colspan="1">29.3</td><td align="left" rowspan="1" colspan="1">7.8</td></tr><tr><td align="left" rowspan="1" colspan="1">D</td><td align="left" rowspan="1" colspan="1">Used colors or fonts that were hard to distinguish</td><td align="left" rowspan="1" colspan="1">0.932</td><td align="left" rowspan="1" colspan="1">34.1</td><td align="left" rowspan="1" colspan="1">49.3</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">8.8</td><td align="left" rowspan="1" colspan="1">1.0</td><td align="left" rowspan="1" colspan="1">1.031</td><td align="left" rowspan="1" colspan="1">35.1</td><td align="left" rowspan="1" colspan="1">30.7</td><td align="left" rowspan="1" colspan="1">21.5</td><td align="left" rowspan="1" colspan="1">8.3</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Did not have enough graphs, pictures, or diagrams</td><td align="left" rowspan="1" colspan="1">0.932</td><td align="left" rowspan="1" colspan="1">33.2</td><td align="left" rowspan="1" colspan="1">50.7</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">2.4</td><td align="left" rowspan="1" colspan="1">0.854</td><td align="left" rowspan="1" colspan="1">37.6</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">17.6</td><td align="left" rowspan="1" colspan="1">2.9</td></tr><tr><td align="left" rowspan="1" colspan="1">C</td><td align="left" rowspan="1" colspan="1">Contained irrelevant graphics, backgrounds, or animations</td><td align="left" rowspan="1" colspan="1">0.980</td><td align="left" rowspan="1" colspan="1">36.6</td><td align="left" rowspan="1" colspan="1">42.4</td><td align="left" rowspan="1" colspan="1">11.2</td><td align="left" rowspan="1" colspan="1">5.9</td><td align="left" rowspan="1" colspan="1">3.9</td><td align="left" rowspan="1" colspan="1">1.005</td><td align="left" rowspan="1" colspan="1">36.6</td><td align="left" rowspan="1" colspan="1">29.3</td><td align="left" rowspan="1" colspan="1">22.4</td><td align="left" rowspan="1" colspan="1">7.3</td></tr><tr><td align="left" rowspan="1" colspan="1">I</td><td align="left" rowspan="1" colspan="1">Varied fonts or colors for no reason</td><td align="left" rowspan="1" colspan="1">1.029</td><td align="left" rowspan="1" colspan="1">33.2</td><td align="left" rowspan="1" colspan="1">46.8</td><td align="left" rowspan="1" colspan="1">7.8</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">3.9</td><td align="left" rowspan="1" colspan="1">0.842</td><td align="left" rowspan="1" colspan="1">40.0</td><td align="left" rowspan="1" colspan="1">35.6</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">4.9</td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Contained graphics that were too complex</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.761</italic></td><td align="left" rowspan="1" colspan="1">43.9</td><td align="left" rowspan="1" colspan="1">41.0</td><td align="left" rowspan="1" colspan="1">11.2</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">1.0</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.758</italic></td><td align="left" rowspan="1" colspan="1">43.4</td><td align="left" rowspan="1" colspan="1">34.1</td><td align="left" rowspan="1" colspan="1">13.7</td><td align="left" rowspan="1" colspan="1">3.4</td></tr><tr><td align="left" rowspan="1" colspan="1">D</td><td align="left" rowspan="1" colspan="1">Contained poor quality images or video</td><td align="left" rowspan="1" colspan="1">0.877</td><td align="left" rowspan="1" colspan="1">38.0</td><td align="left" rowspan="1" colspan="1">46.3</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">5.9</td><td align="left" rowspan="1" colspan="1">2.4</td><td align="left" rowspan="1" colspan="1">0.891</td><td align="left" rowspan="1" colspan="1">37.6</td><td align="left" rowspan="1" colspan="1">35.1</td><td align="left" rowspan="1" colspan="1">15.6</td><td align="left" rowspan="1" colspan="1">5.9</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>WERE ANY OF THE PRESENTATIONS HARD TO UNDERSTAND BECAUSE THE SPEAKERS</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">D</td><td align="left" rowspan="1" colspan="1">Mumbled</td><td align="left" rowspan="1" colspan="1">0.824</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">47.3</td><td align="left" rowspan="1" colspan="1">7.3</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">1.0</td><td align="left" rowspan="1" colspan="1">1.063</td><td align="left" rowspan="1" colspan="1">34.6</td><td align="left" rowspan="1" colspan="1">28.3</td><td align="left" rowspan="1" colspan="1">21.0</td><td align="left" rowspan="1" colspan="1">9.8</td></tr><tr><td align="left" rowspan="1" colspan="1">D</td><td align="left" rowspan="1" colspan="1">Spoke too quietly/did not use the microphone properly</td><td align="left" rowspan="1" colspan="1">0.882</td><td align="left" rowspan="1" colspan="1">37.1</td><td align="left" rowspan="1" colspan="1">45.9</td><td align="left" rowspan="1" colspan="1">8.8</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">1.0</td><td align="left" rowspan="1" colspan="1">1.016</td><td align="left" rowspan="1" colspan="1">33.7</td><td align="left" rowspan="1" colspan="1">33.7</td><td align="left" rowspan="1" colspan="1">17.6</td><td align="left" rowspan="1" colspan="1">8.8</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Did not answer questions</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.719</italic></td><td align="left" rowspan="1" colspan="1">48.8</td><td align="left" rowspan="1" colspan="1">36.6</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">0.791</td><td align="left" rowspan="1" colspan="1">44.4</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">18.0</td><td align="left" rowspan="1" colspan="1">3.4</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Allowed too many questions to interrupt the talk</td><td align="left" rowspan="1" colspan="1">0.931</td><td align="left" rowspan="1" colspan="1">39.5</td><td align="left" rowspan="1" colspan="1">40.0</td><td align="left" rowspan="1" colspan="1">11.2</td><td align="left" rowspan="1" colspan="1">4.9</td><td align="left" rowspan="1" colspan="1">3.9</td><td align="left" rowspan="1" colspan="1">0.932</td><td align="left" rowspan="1" colspan="1">39.0</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">20.0</td><td align="left" rowspan="1" colspan="1">6.3</td></tr><tr><td align="left" rowspan="1" colspan="1">L</td><td align="left" rowspan="1" colspan="1">Did not give the audience time to digest important points</td><td align="left" rowspan="1" colspan="1">1.119</td><td align="left" rowspan="1" colspan="1">23.4</td><td align="left" rowspan="1" colspan="1">53.2</td><td align="left" rowspan="1" colspan="1">12.2</td><td align="left" rowspan="1" colspan="1">6.3</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">1.045</td><td align="left" rowspan="1" colspan="1">30.2</td><td align="left" rowspan="1" colspan="1">38.0</td><td align="left" rowspan="1" colspan="1">22.0</td><td align="left" rowspan="1" colspan="1">6.3</td></tr><tr><td align="left" rowspan="1" colspan="1">C</td><td align="left" rowspan="1" colspan="1">Presented slides with a lot of text and then proceeded to talk while the audience was trying to read</td><td align="left" rowspan="1" colspan="1">1.255</td><td align="left" rowspan="1" colspan="1">23.9</td><td align="left" rowspan="1" colspan="1">45.9</td><td align="left" rowspan="1" colspan="1">13.7</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">1.116</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">37.1</td><td align="left" rowspan="1" colspan="1">21.5</td><td align="left" rowspan="1" colspan="1">9.3</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>DID ANY OF THE SPEAKERS</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">S</td><td align="left" rowspan="1" colspan="1">Read word-for-word from notes or from the slides themselves</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.348</underline></bold></td><td align="left" rowspan="1" colspan="1">19.5</td><td align="left" rowspan="1" colspan="1">48.3</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">15.6</td><td align="left" rowspan="1" colspan="1">3.4</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.315</underline></bold></td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">27.8</td><td align="left" rowspan="1" colspan="1">26.8</td><td align="left" rowspan="1" colspan="1">15.6</td></tr><tr><td align="left" rowspan="1" colspan="1">S</td><td align="left" rowspan="1" colspan="1">Talk to the screen rather than to the audience</td><td align="left" rowspan="1" colspan="1">1.059</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">47.3</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">0.974</td><td align="left" rowspan="1" colspan="1">33.2</td><td align="left" rowspan="1" colspan="1">36.1</td><td align="left" rowspan="1" colspan="1">19.0</td><td align="left" rowspan="1" colspan="1">5.9</td></tr><tr><td align="left" rowspan="1" colspan="1">A</td><td align="left" rowspan="1" colspan="1">Fail to pay attention to audience “feedback” (raised hands, puzzled looks, fidgeting, and signs of boredom)</td><td align="left" rowspan="1" colspan="1">1.142</td><td align="left" rowspan="1" colspan="1">27.8</td><td align="left" rowspan="1" colspan="1">43.4</td><td align="left" rowspan="1" colspan="1">18.0</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">3.4</td><td align="left" rowspan="1" colspan="1">1.081</td><td align="left" rowspan="1" colspan="1">31.7</td><td align="left" rowspan="1" colspan="1">31.7</td><td align="left" rowspan="1" colspan="1">25.9</td><td align="left" rowspan="1" colspan="1">6.8</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Not inject any humor or illustrations to lighten complex material</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><bold><underline>1.299</underline></bold></td><td align="left" rowspan="1" colspan="1">23.9</td><td align="left" rowspan="1" colspan="1">43.4</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">4.4</td><td align="left" rowspan="1" colspan="1">0.899</td><td align="left" rowspan="1" colspan="1">35.6</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">19.0</td><td align="left" rowspan="1" colspan="1">3.4</td></tr><tr><td align="left" rowspan="1" colspan="1">C</td><td align="left" rowspan="1" colspan="1">Use inappropriate or irrelevant jokes or illustrations</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.631</italic></td><td align="left" rowspan="1" colspan="1">50.7</td><td align="left" rowspan="1" colspan="1">39.5</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">1.5</td><td align="left" rowspan="1" colspan="1">2.0</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.691</italic></td><td align="left" rowspan="1" colspan="1">47.8</td><td align="left" rowspan="1" colspan="1">28.8</td><td align="left" rowspan="1" colspan="1">14.1</td><td align="left" rowspan="1" colspan="1">2.4</td></tr><tr><td colspan="13" style="background-color:gray" align="left" rowspan="1"><bold>WERE ANY OF THE SPEAKERS UNPREPARED, IN THAT THEY</bold></td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Stumbled over their presentation, repeated themselves, seemed puzzled by their own slides</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.751</italic></td><td align="left" rowspan="1" colspan="1">42.4</td><td align="left" rowspan="1" colspan="1">43.9</td><td align="left" rowspan="1" colspan="1">10.2</td><td align="left" rowspan="1" colspan="1">2.9</td><td align="left" rowspan="1" colspan="1">0.5</td><td align="left" rowspan="1" colspan="1">0.867</td><td align="left" rowspan="1" colspan="1">41.0</td><td align="left" rowspan="1" colspan="1">34.1</td><td align="left" rowspan="1" colspan="1">12.7</td><td align="left" rowspan="1" colspan="1">7.8</td></tr><tr><td align="left" rowspan="1" colspan="1">I</td><td align="left" rowspan="1" colspan="1">Had mistakes in their slides</td><td align="left" rowspan="1" colspan="1">0.907</td><td align="left" rowspan="1" colspan="1">32.2</td><td align="left" rowspan="1" colspan="1">52.7</td><td align="left" rowspan="1" colspan="1">8.3</td><td align="left" rowspan="1" colspan="1">5.9</td><td align="left" rowspan="1" colspan="1">1.0</td><td align="left" rowspan="1" colspan="1">0.862</td><td align="left" rowspan="1" colspan="1">39.0</td><td align="left" rowspan="1" colspan="1">33.7</td><td align="left" rowspan="1" colspan="1">19.0</td><td align="left" rowspan="1" colspan="1">3.4</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Went over the allotted time (or rushed to finish in time)</td><td align="left" rowspan="1" colspan="1">1.186</td><td align="left" rowspan="1" colspan="1">25.9</td><td align="left" rowspan="1" colspan="1">44.9</td><td align="left" rowspan="1" colspan="1">15.1</td><td align="left" rowspan="1" colspan="1">11.7</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">1.117</td><td align="left" rowspan="1" colspan="1">27.3</td><td align="left" rowspan="1" colspan="1">39.5</td><td align="left" rowspan="1" colspan="1">20.0</td><td align="left" rowspan="1" colspan="1">9.3</td></tr><tr><td align="left" rowspan="1" colspan="1">R</td><td align="left" rowspan="1" colspan="1">Did not have enough material to fill the time</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.815</italic></td><td align="left" rowspan="1" colspan="1">41.0</td><td align="left" rowspan="1" colspan="1">45.4</td><td align="left" rowspan="1" colspan="1">6.8</td><td align="left" rowspan="1" colspan="1">4.9</td><td align="left" rowspan="1" colspan="1">2.0</td><td colspan="1" align="left" style="background-color:gray" rowspan="1"><italic>0.600</italic></td><td align="left" rowspan="1" colspan="1">51.2</td><td align="left" rowspan="1" colspan="1">29.8</td><td align="left" rowspan="1" colspan="1">9.3</td><td align="left" rowspan="1" colspan="1">2.4</td></tr><tr><td align="left" rowspan="1" colspan="1">Tech</td><td align="left" rowspan="1" colspan="1">Did not know how to use their technology (software, computer, projector, etc.)</td><td align="left" rowspan="1" colspan="1">0.854</td><td align="left" rowspan="1" colspan="1">38.5</td><td align="left" rowspan="1" colspan="1">46.8</td><td align="left" rowspan="1" colspan="1">7.3</td><td align="left" rowspan="1" colspan="1">5.4</td><td align="left" rowspan="1" colspan="1">2.0</td><td align="left" rowspan="1" colspan="1">0.854</td><td align="left" rowspan="1" colspan="1">42.9</td><td align="left" rowspan="1" colspan="1">32.2</td><td align="left" rowspan="1" colspan="1">14.1</td><td align="left" rowspan="1" colspan="1">7.3</td></tr></tbody></table><table-wrap-foot><p><italic>A, appropriate knowledge; C, compatibility; D, discriminability; I, informative change; L, limited capacity; R, relevance; S, salience; Tech, problems with the technology</italic>.</p></table-wrap-foot></table-wrap></app></app-group><ref-list><title>References</title><ref id="B1"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Allen</surname><given-names>R. C.</given-names></name><name><surname>Rubin</surname><given-names>M. L.</given-names></name></person-group> (<year>1981</year>). <article-title>Chromostereopsis</article-title>. <source>Surv. Ophthalmol.</source><volume>26</volume>, <fpage>22</fpage>–<lpage>27</lpage><pub-id pub-id-type="doi">10.1016/0039-6257(81)90121-1</pub-id><pub-id pub-id-type="pmid">7280992</pub-id></mixed-citation></ref><ref id="B2"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Alvarez</surname><given-names>G.</given-names></name><name><surname>Franconeri</surname><given-names>S.</given-names></name></person-group> (<year>2007</year>). <article-title>How many objects can you track? Evidence for a resource-limited attentive tracking mechanism</article-title>. <source>J. Vis.</source><volume>7</volume>, <fpage>1</fpage>–<lpage>10</lpage><pub-id pub-id-type="doi">10.1167/7.12.1</pub-id></mixed-citation></ref><ref id="B3"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Apperson</surname><given-names>J. M.</given-names></name><name><surname>Laws</surname><given-names>E. L.</given-names></name><name><surname>Scepansky</surname><given-names>J. A.</given-names></name></person-group> (<year>2006</year>). <article-title>The impact of presentation graphics on students’ experience in the classroom</article-title>. <source>Comput. Educ.</source><volume>47</volume>, <fpage>116</fpage>–<lpage>126</lpage><pub-id pub-id-type="doi">10.1016/j.compedu.2004.09.003</pub-id></mixed-citation></ref><ref id="B4"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Apperson</surname><given-names>J. M.</given-names></name><name><surname>Laws</surname><given-names>E. L.</given-names></name><name><surname>Scepansky</surname><given-names>J. A.</given-names></name></person-group> (<year>2008</year>). <article-title>An assessment of student preferences for PowerPoint presentation structure in undergraduate courses</article-title>. <source>Comput. Educ.</source><volume>50</volume>, <fpage>148</fpage>–<lpage>153</lpage><pub-id pub-id-type="doi">10.1016/j.compedu.2006.04.003</pub-id></mixed-citation></ref><ref id="B5"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Aspillaga</surname><given-names>M.</given-names></name></person-group> (<year>1996</year>). <article-title>Perceptual foundations in the design of visual displays</article-title>. <source>Comput. Human Behav.</source><volume>12</volume>, <fpage>587</fpage>–<lpage>600</lpage><pub-id pub-id-type="doi">10.1016/S0747-5632(96)00028-3</pub-id></mixed-citation></ref><ref id="B6"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Baddeley</surname><given-names>A. D.</given-names></name></person-group> (<year>2007</year>). <source>Working Memory, Thought and Action</source>. <publisher-loc>Oxford</publisher-loc>: <publisher-name>Oxford University Press</publisher-name></mixed-citation></ref><ref id="B7"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bartsch</surname><given-names>R. A.</given-names></name><name><surname>Cobern</surname><given-names>K. M.</given-names></name></person-group> (<year>2003</year>). <article-title>Effectiveness of PowerPoint presentations in lectures</article-title>. <source>Comput. Educ.</source><volume>41</volume>, <fpage>77</fpage>–<lpage>86</lpage><pub-id pub-id-type="doi">10.1016/S0360-1315(03)00027-7</pub-id></mixed-citation></ref><ref id="B8"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Beets</surname><given-names>S. D.</given-names></name><name><surname>Lobingier</surname><given-names>P. G.</given-names></name></person-group> (<year>2001</year>). <article-title>Pedagogical techniques: student performance and preferences</article-title>. <source>J. Educ. Bus.</source><volume>76</volume>, <fpage>231</fpage>–<lpage>235</lpage><pub-id pub-id-type="doi">10.1080/08832320109601316</pub-id></mixed-citation></ref><ref id="B9"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Buchel</surname><given-names>C.</given-names></name><name><surname>Josephs</surname><given-names>O.</given-names></name><name><surname>Rees</surname><given-names>G.</given-names></name><name><surname>Turner</surname><given-names>R.</given-names></name><name><surname>Frith</surname><given-names>C. D.</given-names></name><name><surname>Friston</surname><given-names>K. J.</given-names></name></person-group> (<year>1998</year>). <article-title>The functional anatomy of attention to visual motion – a functional MRI study</article-title>. <source>Brain</source><volume>121</volume>, <fpage>1281</fpage>–<lpage>1294</lpage><pub-id pub-id-type="doi">10.1093/brain/121.7.1193</pub-id><pub-id pub-id-type="pmid">9679780</pub-id></mixed-citation></ref><ref id="B10"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Campbell</surname><given-names>F. A.</given-names></name><name><surname>Goldman</surname><given-names>B. D.</given-names></name><name><surname>Boccia</surname><given-names>M. L.</given-names></name><name><surname>Skinner</surname><given-names>M.</given-names></name></person-group> (<year>2004</year>). <article-title>The effect of format modifications and reading comprehension on recall of informed consent information by low-income parents: a comparison of print, video, and computer-based presentations</article-title>. <source>Patient Educ. Couns.</source><volume>53</volume>, <fpage>205</fpage>–<lpage>216</lpage><pub-id pub-id-type="doi">10.1016/S0738-3991(03)00162-9</pub-id><pub-id pub-id-type="pmid">15140461</pub-id></mixed-citation></ref><ref id="B11"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Cowan</surname><given-names>N.</given-names></name></person-group> (<year>2001</year>). <article-title>The magical number 4 in short-term memory: a reconsideration of mental storage capacity</article-title>. <source>Behav. Brain Sci.</source><volume>24</volume>, <fpage>87</fpage>–<lpage>114</lpage><pub-id pub-id-type="doi">10.1017/S0140525X0161392X</pub-id><pub-id pub-id-type="pmid">11515286</pub-id></mixed-citation></ref><ref id="B12"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>De Valois</surname><given-names>R. L.</given-names></name><name><surname>De Valois</surname><given-names>K. K.</given-names></name></person-group> (<year>1990</year>). <source>Spatial Vision</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Oxford University Press</publisher-name></mixed-citation></ref><ref id="B13"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Farkas</surname><given-names>D. K.</given-names></name></person-group> (<year>2006</year>). <article-title>Toward a better understanding of PowerPoint deck design</article-title>. <source>Inform. Des. J.</source><volume>14</volume>, <fpage>162</fpage>–<lpage>171</lpage></mixed-citation></ref><ref id="B14"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Fleming</surname><given-names>M.</given-names></name><name><surname>Levie</surname><given-names>W. H.</given-names></name></person-group> (<year>1993</year>). <source>Instructional Message Design: Principles from the Behavioral and Cognitive Sciences</source>, <edition>2nd Edn.</edition><publisher-loc>Englewood Cliffs, NJ</publisher-loc>: <publisher-name>Educational Technology Publications</publisher-name></mixed-citation></ref><ref id="B15"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Glimcher</surname><given-names>P. W.</given-names></name><name><surname>Sparks</surname><given-names>D. L.</given-names></name></person-group> (<year>1992</year>). <article-title>Movement selection in advance of action in the superior colliculus</article-title>. <source>Nature</source><volume>355</volume>, <fpage>542</fpage>–<lpage>545</lpage><pub-id pub-id-type="doi">10.1038/355542a0</pub-id><pub-id pub-id-type="pmid">1741032</pub-id></mixed-citation></ref><ref id="B16"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Grice</surname><given-names>H. P.</given-names></name></person-group> (<year>1975</year>). <article-title>“Logic and conversation,”</article-title> in <source>Syntax and Semantics</source>, Vol. <volume>3</volume>, eds <person-group person-group-type="editor"><name><surname>Cole</surname><given-names>P.</given-names></name><name><surname>Morgan</surname><given-names>J.</given-names></name></person-group> (<publisher-loc>New York</publisher-loc>: <publisher-name>Academic Press</publisher-name>), <fpage>41</fpage>–<lpage>58</lpage></mixed-citation></ref><ref id="B17"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Gropper</surname><given-names>G. L.</given-names></name></person-group> (<year>1991</year>). <source>Text Displays Analysis and Systematic Design</source>. <publisher-loc>Englewood Cliffs, NJ</publisher-loc>: <publisher-name>Educational Technology Publications</publisher-name></mixed-citation></ref><ref id="B18"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gunel</surname><given-names>M.</given-names></name><name><surname>Hand</surname><given-names>B.</given-names></name><name><surname>Gunduz</surname><given-names>S.</given-names></name></person-group> (<year>2006</year>). <article-title>Comparing student understanding of quantum physics when embedding multimodal representations into two different writing formats: presentation format versus summary report format</article-title>. <source>Sci. Educ.</source><volume>90</volume>, <fpage>1093</fpage>–<lpage>1112</lpage></mixed-citation></ref><ref id="B19"><mixed-citation publication-type="book"><person-group person-group-type="editor"><name><surname>Helander</surname><given-names>M.</given-names></name><name><surname>Landauer</surname><given-names>T.</given-names></name><name><surname>Prabhu</surname><given-names>P.</given-names></name></person-group> (eds). (<year>1997</year>). <source>Handbook of Human-Computer Interaction</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Elsevier Sciences</publisher-name></mixed-citation></ref><ref id="B20"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Held</surname><given-names>R.</given-names></name></person-group> (<year>1980</year>). <article-title>“The rediscovery of adaptability in the visual system: effects of extrinsic and intrinsic chromatic dispersion,”</article-title> in <source>Visual Coding and Adaptability</source>, ed. <person-group person-group-type="editor"><name><surname>Harris</surname><given-names>C. S.</given-names></name></person-group> (<publisher-loc>Hillsdale, NJ</publisher-loc>: <publisher-name>Erlbaum</publisher-name>), <fpage>69</fpage>–<lpage>94</lpage></mixed-citation></ref><ref id="B21"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hu</surname><given-names>A.</given-names></name><name><surname>Wilson</surname><given-names>T.</given-names></name><name><surname>Ladak</surname><given-names>H.</given-names></name><name><surname>Haase</surname><given-names>P.</given-names></name><name><surname>Fung</surname><given-names>K.</given-names></name></person-group> (<year>2009</year>). <article-title>Three-dimensional educational computer model of the larynx: voicing a new direction</article-title>. <source>Arch. Otolaryngol. Head Neck Surg.</source><volume>135</volume>, <fpage>677</fpage>–<lpage>681</lpage><pub-id pub-id-type="doi">10.1001/archoto.2009.68</pub-id><pub-id pub-id-type="pmid">19620589</pub-id></mixed-citation></ref><ref id="B22"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Intriligator</surname><given-names>J.</given-names></name><name><surname>Cavanagh</surname><given-names>P.</given-names></name></person-group> (<year>2001</year>). <article-title>The spatial resolution of visual attention</article-title>. <source>Cogn. Psychol.</source><volume>43</volume>, <fpage>171</fpage>–<lpage>216</lpage><pub-id pub-id-type="doi">10.1006/cogp.2001.0755</pub-id><pub-id pub-id-type="pmid">11689021</pub-id></mixed-citation></ref><ref id="B23"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Karns</surname><given-names>T. E.</given-names></name><name><surname>Irvin</surname><given-names>S. J.</given-names></name><name><surname>Suranic</surname><given-names>S. L.</given-names></name><name><surname>Rivardo</surname><given-names>M. G.</given-names></name></person-group> (<year>2009</year>). <article-title>Collaborative recall reduces the effect of a misleading post event narrative</article-title>. <source>N. Am. J. Psychol.</source><volume>11</volume>, <fpage>17</fpage>–<lpage>28</lpage></mixed-citation></ref><ref id="B24"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kosslyn</surname><given-names>S. M.</given-names></name></person-group> (<year>2006</year>). <source>Graph Design for the Eye and Mind</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Oxford University Press</publisher-name></mixed-citation></ref><ref id="B25"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kosslyn</surname><given-names>S. M.</given-names></name></person-group> (<year>2007</year>). <source>Clear and to the Point: Eight Psychological Principles for Compelling PowerPoint<sup>®</sup> Presentations</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Oxford University Press</publisher-name></mixed-citation></ref><ref id="B26"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kosslyn</surname><given-names>S. M.</given-names></name></person-group> (<year>2010</year>). <source>Better PowerPoint<sup>®</sup></source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Oxford University Press</publisher-name></mixed-citation></ref><ref id="B27"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Krauzlis</surname><given-names>R. J.</given-names></name><name><surname>Liston</surname><given-names>D.</given-names></name><name><surname>Carello</surname><given-names>C. D.</given-names></name></person-group> (<year>2004</year>). <article-title>Target selection and the superior colliculus: goals, choices and hypotheses</article-title>. <source>Vision Res.</source><volume>44</volume>, <fpage>1445</fpage>–<lpage>1451</lpage><pub-id pub-id-type="doi">10.1016/j.visres.2004.01.005</pub-id><pub-id pub-id-type="pmid">15066403</pub-id></mixed-citation></ref><ref id="B28"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Larkin</surname><given-names>J. H.</given-names></name><name><surname>Simon</surname><given-names>H. A.</given-names></name></person-group> (<year>1987</year>). <article-title>Why a diagram is (sometimes) worth ten thousand words</article-title>. <source>Cogn. Sci.</source><volume>11</volume>, <fpage>65</fpage>–<lpage>99</lpage><pub-id pub-id-type="doi">10.1111/j.1551-6708.1987.tb00863.x</pub-id></mixed-citation></ref><ref id="B29"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Levasseur</surname><given-names>D. G.</given-names></name><name><surname>Sawyer</surname><given-names>D. K.</given-names></name></person-group> (<year>2006</year>). <article-title>Pedagogy meets PowerPoint: a research review of the effects of computer-generated slides in the classroom</article-title>. <source>Rev. Commun.</source><volume>6</volume>, <fpage>101</fpage>–<lpage>123</lpage><pub-id pub-id-type="doi">10.1080/15358590600763383</pub-id></mixed-citation></ref><ref id="B30"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Lund</surname><given-names>A. M.</given-names></name></person-group> (<year>1997</year>). <article-title>Expert ratings of usability maxims</article-title>. <source>Ergon. Des.</source><volume>5</volume>, <fpage>15</fpage>–<lpage>20</lpage></mixed-citation></ref><ref id="B31"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>MacLeod</surname><given-names>C. M.</given-names></name></person-group> (<year>1991</year>). <article-title>Half a century of research on the Stroop effect: an integrative review</article-title>. <source>Psychol. Rev.</source><volume>109</volume>, <fpage>163</fpage>–<lpage>203</lpage></mixed-citation></ref><ref id="B32"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Macmillan</surname><given-names>N. A.</given-names></name><name><surname>Ornstein</surname><given-names>A. S.</given-names></name></person-group> (<year>1998</year>). <article-title>The mean-integral representation of rectangles</article-title>. <source>Percept. Psychophys.</source><volume>60</volume>, <fpage>250</fpage>–<lpage>262</lpage><pub-id pub-id-type="doi">10.3758/BF03206034</pub-id><pub-id pub-id-type="pmid">9529909</pub-id></mixed-citation></ref><ref id="B33"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Mahar</surname><given-names>S.</given-names></name><name><surname>Yaylacicegi</surname><given-names>U.</given-names></name><name><surname>Janicki</surname><given-names>T.</given-names></name></person-group> (<year>2009</year>). <article-title>The dark side of custom animation</article-title>. <source>Int. J. Innovat. Learn.</source><volume>6</volume>, <fpage>581</fpage>–<lpage>592</lpage><pub-id pub-id-type="doi">10.1504/IJIL.2009.026645</pub-id></mixed-citation></ref><ref id="B34"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Mejdal</surname><given-names>S.</given-names></name><name><surname>McCauley</surname><given-names>M. E.</given-names></name><name><surname>Beringer</surname><given-names>D. B.</given-names></name></person-group> (<year>2001</year>). <source>Human Factors Design Guidelines for Multifunction Displays</source>. <publisher-loc>Springfield, VA</publisher-loc>: <publisher-name>National Technical Information Service</publisher-name></mixed-citation></ref><ref id="B35"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Osman</surname><given-names>M. E.</given-names></name><name><surname>Hannafin</surname><given-names>M. J.</given-names></name></person-group> (<year>1992</year>). <article-title>Metacognition research and theory: Analysis and implications for instructional design</article-title>. <source>Educ. Technol. Res. Dev.</source><volume>40</volume>, <fpage>83</fpage>–<lpage>99</lpage><pub-id pub-id-type="doi">10.1007/BF02297053</pub-id></mixed-citation></ref><ref id="B36"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Palmer</surname><given-names>S. E.</given-names></name></person-group> (<year>1992</year>). <article-title>“Modern theories of Gestalt perception,”</article-title> in <source>Understanding Vision</source>, ed. <person-group person-group-type="editor"><name><surname>Humphreys</surname><given-names>G. W.</given-names></name></person-group> (<publisher-loc>Oxford</publisher-loc>: <publisher-name>Blackwell</publisher-name>), <fpage>39</fpage>–<lpage>70</lpage></mixed-citation></ref><ref id="B37"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Parker</surname><given-names>I.</given-names></name></person-group> (<year>2001</year>). <article-title>Absolute PowerPoint: can a software package edit our thoughts?</article-title><source>New Yorker</source><volume>76</volume>, <fpage>76</fpage>–<lpage>87</lpage></mixed-citation></ref><ref id="B38"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Posner</surname><given-names>M. I.</given-names></name></person-group> (<year>1980</year>). <article-title>Orienting of attention</article-title>. <source>Q. J. Exp. Psychol.</source><volume>32</volume>, <fpage>3</fpage>–<lpage>25</lpage><pub-id pub-id-type="doi">10.1080/00335558008248231</pub-id><pub-id pub-id-type="pmid">7367577</pub-id></mixed-citation></ref><ref id="B39"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Potter</surname><given-names>M. C.</given-names></name></person-group> (<year>1966</year>). <article-title>“On perceptual recognition,”</article-title> in <source>Studies in Cognitive Growth</source>, eds <person-group person-group-type="editor"><name><surname>Bruner</surname><given-names>J. S.</given-names></name><name><surname>Olver</surname><given-names>R. A.</given-names></name><name><surname>Greenfield</surname><given-names>P. M.</given-names></name></person-group> (<publisher-loc>New York</publisher-loc>: <publisher-name>Wiley</publisher-name>), <fpage>103</fpage>–<lpage>134</lpage></mixed-citation></ref><ref id="B40"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Reisberg</surname><given-names>D.</given-names></name></person-group> (<year>2006</year>). <source>Cognition: Exploring the Science of the Mind, Third Edition</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>W.W. Norton</publisher-name></mixed-citation></ref><ref id="B41"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rosch</surname><given-names>E.</given-names></name><name><surname>Simpson</surname><given-names>C.</given-names></name><name><surname>Miller</surname><given-names>R. S.</given-names></name></person-group> (<year>1976</year>). <article-title>Structural bases of typicality effects</article-title>. <source>J. Exp. Psychol. Hum. Percept. Perform.</source><volume>2</volume>, <fpage>491</fpage>–<lpage>502</lpage><pub-id pub-id-type="doi">10.1037/0096-1523.2.4.491</pub-id></mixed-citation></ref><ref id="B42"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Savoy</surname><given-names>A.</given-names></name><name><surname>Proctor</surname><given-names>R. W.</given-names></name><name><surname>Salvendy</surname><given-names>G.</given-names></name></person-group> (<year>2009</year>). <article-title>Information retention from PowerPoint™ and traditional lectures</article-title>. <source>Comput. Educ.</source><volume>52</volume>, <fpage>858</fpage>–<lpage>867</lpage><pub-id pub-id-type="doi">10.1016/j.compedu.2008.12.005</pub-id></mixed-citation></ref><ref id="B43"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Schmidt</surname><given-names>S. R.</given-names></name></person-group> (<year>1991</year>). <article-title>Can we have a distinctive theory of memory?</article-title><source>Mem. Cognit.</source><volume>19</volume>, <fpage>523</fpage>–<lpage>542</lpage><pub-id pub-id-type="doi">10.3758/BF03197149</pub-id><pub-id pub-id-type="pmid">1758300</pub-id></mixed-citation></ref><ref id="B44"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Schwartz</surname><given-names>N. H.</given-names></name><name><surname>Ellsworth</surname><given-names>L. S.</given-names></name><name><surname>Graham</surname><given-names>L.</given-names></name><name><surname>Knight</surname><given-names>B.</given-names></name></person-group> (<year>1998</year>). <article-title>Accessing prior knowledge to remember text: a comparison of advance organizers and maps</article-title>. <source>Contemp. Educ. Psychol.</source><volume>23</volume>, <fpage>65</fpage>–<lpage>89</lpage><pub-id pub-id-type="doi">10.1006/ceps.1997.0958</pub-id><pub-id pub-id-type="pmid">9514689</pub-id></mixed-citation></ref><ref id="B45"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Shannon</surname><given-names>C. E.</given-names></name></person-group> (<year>1948</year>). <article-title>A mathematical theory of communication</article-title>. <source>Bell Syst. Tech. J.</source><volume>27</volume>, <fpage>379</fpage>–<lpage>423</lpage>; 623–656.</mixed-citation></ref><ref id="B46"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Shneiderman</surname><given-names>B.</given-names></name></person-group> (<year>1992</year>). <source>Designing the User Interface: Strategies for Effective Human-Computer Interaction</source>, <edition>2nd Edn.</edition><publisher-loc>Reading, MA</publisher-loc>: <publisher-name>Addison-Wesley</publisher-name></mixed-citation></ref><ref id="B47"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Smith</surname><given-names>E. E.</given-names></name><name><surname>Kosslyn</surname><given-names>S. M.</given-names></name></person-group> (<year>2006</year>). <source>Cognitive Psychology: Mind and Brain</source>. <publisher-loc>New York, NY</publisher-loc>: <publisher-name>Prentice Hall</publisher-name></mixed-citation></ref><ref id="B48"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Smith</surname><given-names>S.</given-names></name><name><surname>Mosier</surname><given-names>J.</given-names></name></person-group> (<year>1986</year>). <source>Guidelines for Designing Interface Software</source>. <publisher-loc>Springfield, VA</publisher-loc>: <publisher-name>National Technical Information Services</publisher-name></mixed-citation></ref><ref id="B49"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Speier</surname><given-names>C.</given-names></name></person-group> (<year>2006</year>). <article-title>The influence of information presentation formats on complex task decision-making performance</article-title>. <source>Int. J. Hum. Comput. Stud.</source><volume>64</volume>, <fpage>1115</fpage>–<lpage>1131</lpage><pub-id pub-id-type="doi">10.1016/j.ijhcs.2006.06.007</pub-id></mixed-citation></ref><ref id="B50"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stanney</surname><given-names>K.</given-names></name><name><surname>Samman</surname><given-names>S.</given-names></name><name><surname>Reeves</surname><given-names>L.</given-names></name><name><surname>Hale</surname><given-names>K.</given-names></name><name><surname>Buff</surname><given-names>W.</given-names></name><name><surname>Bowers</surname><given-names>C.</given-names></name><name><surname>Goldiez</surname><given-names>B.</given-names></name><name><surname>Nicholson</surname><given-names>D.</given-names></name><name><surname>Lackey</surname><given-names>S.</given-names></name></person-group> (<year>2004</year>). <article-title>A paradigm shift in interactive computing: deriving multimodal design principles from behavioral and neurological foundations</article-title>. <source>Int. J. Hum. Comput. Interact.</source><volume>17</volume>, <fpage>229</fpage>–<lpage>257</lpage><pub-id pub-id-type="doi">10.1207/s15327590ijhc1702_7</pub-id></mixed-citation></ref><ref id="B51"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Stevens</surname><given-names>S. S.</given-names></name></person-group> (<year>1975</year>). <source>Psychophysics</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Wiley</publisher-name></mixed-citation></ref><ref id="B52"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stoner</surname><given-names>M. R.</given-names></name></person-group> (<year>2007</year>). <article-title>PowerPoint in a new key</article-title>. <source>Commun. Educ.</source><volume>56</volume>, <fpage>354</fpage>–<lpage>381</lpage><pub-id pub-id-type="doi">10.1080/03634520701342052</pub-id></mixed-citation></ref><ref id="B53"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Stromeyer</surname><given-names>C. F.</given-names><suffix>III</suffix></name><name><surname>Klein</surname><given-names>S.</given-names></name></person-group> (<year>1974</year>). <article-title>Spatial frequency channels in human vision as asymmetric (edge) mechanisms</article-title>. <source>Vision Res.</source><volume>14</volume>, <fpage>1409</fpage>–<lpage>1420</lpage><pub-id pub-id-type="doi">10.1016/0042-6989(74)90016-9</pub-id><pub-id pub-id-type="pmid">4446371</pub-id></mixed-citation></ref><ref id="B54"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Suchoff</surname><given-names>I. B.</given-names></name></person-group> (<year>2006</year>). <article-title>Editorial: the power & abuse of Microsoft PowerPoint</article-title>. <source>J. Behav. Optom.</source><volume>17</volume>, <fpage>30</fpage>, 46.</mixed-citation></ref><ref id="B55"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Susskind</surname><given-names>J. E.</given-names></name></person-group> (<year>2005</year>). <article-title>PowerPoint’s power in the classroom: enhancing students’ self-efficacy and attitudes</article-title>. <source>Comput. Educ.</source><volume>45</volume>, <fpage>203</fpage>–<lpage>215</lpage><pub-id pub-id-type="doi">10.1016/j.compedu.2004.07.005</pub-id></mixed-citation></ref><ref id="B56"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Susskind</surname><given-names>J. E.</given-names></name></person-group> (<year>2008</year>). <article-title>Limits of PowerPoint’s power: enhancing students’ self-efficacy and attitudes but not their behavior</article-title>. <source>Comput. Educ.</source><volume>50</volume>, <fpage>1228</fpage>–<lpage>1239</lpage><pub-id pub-id-type="doi">10.1016/j.compedu.2006.12.001</pub-id></mixed-citation></ref><ref id="B57"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Sweller</surname><given-names>J.</given-names></name></person-group> (<year>1999</year>). <source>Instructional Design in Technical Areas</source>. <publisher-loc>Camberwell, VIC</publisher-loc>: <publisher-name>Australian Council for Educational Research</publisher-name></mixed-citation></ref><ref id="B58"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Sweller</surname><given-names>J.</given-names></name><name><surname>van Merrienboer</surname><given-names>J. J. G.</given-names></name><name><surname>Paas</surname><given-names>F. G. W. C.</given-names></name></person-group> (<year>1998</year>). <article-title>Cognitive architecture and instructional design</article-title>. <source>Educ. Psychol. Rev.</source><volume>10</volume>, <fpage>251</fpage>–<lpage>296</lpage><pub-id pub-id-type="doi">10.1023/A:1022193728205</pub-id></mixed-citation></ref><ref id="B59"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Szabo</surname><given-names>A.</given-names></name><name><surname>Hastings</surname><given-names>N.</given-names></name></person-group> (<year>2000</year>). <article-title>Using IT in the undergraduate classroom: should we replace the blackboard with PowerPoint?</article-title><source>Comput. Educ.</source><volume>35</volume>, <fpage>175</fpage>–<lpage>187</lpage><pub-id pub-id-type="doi">10.1016/S0360-1315(00)00030-0</pub-id></mixed-citation></ref><ref id="B60"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Taylor</surname><given-names>J. S.</given-names></name><name><surname>Dube</surname><given-names>C. E.</given-names></name><name><surname>Pipas</surname><given-names>C. F.</given-names></name><name><surname>Fuller</surname><given-names>B. K.</given-names></name><name><surname>Lavallee</surname><given-names>L. K.</given-names></name><name><surname>Rosen</surname><given-names>R.</given-names></name></person-group> (<year>2004</year>). <article-title>Teaching the testicular exam: a model curriculum from “A” to “Zack”</article-title>. <source>Fam. Med.</source><volume>36</volume>, <fpage>209</fpage>–<lpage>213</lpage><pub-id pub-id-type="pmid">14999579</pub-id></mixed-citation></ref><ref id="B61"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Thomas</surname><given-names>J. P.</given-names></name><name><surname>Gille</surname><given-names>J.</given-names></name></person-group> (<year>1979</year>). <article-title>Bandwidths of orientation channels in human vision</article-title>. <source>J. Opt. Soc. Am.</source><volume>69</volume>, <fpage>652</fpage>–<lpage>660</lpage><pub-id pub-id-type="doi">10.1364/JOSA.69.000652</pub-id><pub-id pub-id-type="pmid">479968</pub-id></mixed-citation></ref><ref id="B62"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Travis</surname><given-names>D.</given-names></name></person-group> (<year>1991</year>). <source>Effective Color Displays: Theory and Practice</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Academic Press</publisher-name></mixed-citation></ref><ref id="B63"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Tversky</surname><given-names>B.</given-names></name><name><surname>Morrsion</surname><given-names>J. B.</given-names></name><name><surname>Betrancourt</surname><given-names>M.</given-names></name></person-group> (<year>2002</year>). <article-title>Animation: can it facilitate?</article-title><source>Int. J. Hum. Comput. Stud.</source><volume>57</volume>, <fpage>247</fpage>–<lpage>262</lpage><pub-id pub-id-type="doi">10.1006/ijhc.2002.1017</pub-id></mixed-citation></ref><ref id="B64"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Ungerleider</surname><given-names>L. G.</given-names></name><name><surname>Mishkin</surname><given-names>M.</given-names></name></person-group> (<year>1982</year>). <article-title>“Two cortical visual systems,”</article-title> in <source>Analysis of Visual Behavior</source>, eds <person-group person-group-type="editor"><name><surname>Ingle</surname><given-names>D. J.</given-names></name><name><surname>Goodale</surname><given-names>M. A.</given-names></name><name><surname>Mansfield</surname><given-names>R. J.</given-names></name></person-group> (<publisher-loc>Cambridge, MA</publisher-loc>: <publisher-name>The MIT Press</publisher-name>), <fpage>549</fpage>–<lpage>586</lpage></mixed-citation></ref><ref id="B65"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vekiri</surname><given-names>I.</given-names></name></person-group> (<year>2002</year>). <article-title>What is the value of graphical displays in learning?</article-title><source>Educ. Psychol. Rev.</source><volume>14</volume>, <fpage>261</fpage>–<lpage>312</lpage><pub-id pub-id-type="doi">10.1023/A:1016064429161</pub-id></mixed-citation></ref><ref id="B66"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vessey</surname><given-names>I.</given-names></name></person-group> (<year>1991</year>). <article-title>Cognitive fit: a theory-based analysis of the graphs versus tables literature</article-title>. <source>Decis. Sci.</source><volume>22</volume>, <fpage>219</fpage>–<lpage>240</lpage><pub-id pub-id-type="doi">10.1111/j.1540-5915.1991.tb00344.x</pub-id></mixed-citation></ref><ref id="B67"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wagoner</surname><given-names>B.</given-names></name></person-group> (<year>2008</year>). <article-title>Narrative form and content in remembering</article-title>. <source>Integr. Psychol. Behav. Sci.</source><volume>42</volume>, <fpage>315</fpage>–<lpage>323</lpage><pub-id pub-id-type="doi">10.1007/s12124-008-9077-4</pub-id><pub-id pub-id-type="pmid">18769986</pub-id></mixed-citation></ref><ref id="B68"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Watzman</surname><given-names>S.</given-names></name></person-group> (<year>2003</year>). <article-title>“Visual design principles for usable interfaces,”</article-title> in <source>The Human-Computer Interaction Handbook: Fundamentals, Evolving Technologies, and Emerging Applications</source>, eds <person-group person-group-type="editor"><name><surname>Jacko</surname><given-names>J. A.</given-names></name><name><surname>Sears</surname><given-names>A.</given-names></name></person-group> (<publisher-loc>Mahwah, NJ</publisher-loc>: <publisher-name>Lawrence Erlbaum Associates</publisher-name>), <fpage>263</fpage>–<lpage>285</lpage></mixed-citation></ref><ref id="B69"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Wittgenstein</surname><given-names>L.</given-names></name></person-group> (<year>1953</year>/2001). <source>Philosophical Investigations</source>. <publisher-loc>Oxford</publisher-loc>: <publisher-name>Blackwell Publishing</publisher-name></mixed-citation></ref><ref id="B70"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Wolfe</surname><given-names>J. M.</given-names></name></person-group> (<year>1998</year>). <article-title>“Visual search,”</article-title> in <source>Attention</source>, ed. <person-group person-group-type="editor"><name><surname>Pashler</surname><given-names>H.</given-names></name></person-group> (<publisher-loc>East Sussex</publisher-loc>: <publisher-name>Psychology Press</publisher-name>), <fpage>13</fpage>–<lpage>56</lpage></mixed-citation></ref><ref id="B71"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wong</surname><given-names>M. S.</given-names></name><name><surname>Lemaire</surname><given-names>E. D.</given-names></name><name><surname>Leung</surname><given-names>A. K. L.</given-names></name><name><surname>Chan</surname><given-names>M. F.</given-names></name></person-group> (<year>2004</year>). <article-title>Enhancement of prosthetics and orthotics learning and teaching through e-Learning technology and methodology</article-title>. <source>Prosthet. Orthot. Int.</source><volume>28</volume>, <fpage>55</fpage>–<lpage>59</lpage><pub-id pub-id-type="pmid">15171579</pub-id></mixed-citation></ref><ref id="B72"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Woodson</surname><given-names>W.</given-names></name><name><surname>Tillman</surname><given-names>B.</given-names></name><name><surname>Tillman</surname><given-names>P.</given-names></name></person-group> (<year>1992</year>). <source>Human Factors Design Handbook</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>McGraw-Hill</publisher-name></mixed-citation></ref></ref-list></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Terre/explor/CobaltMaghrebV1/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 0004169 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 0004169 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Terre
|area= CobaltMaghrebV1
|flux= Pmc
|étape= Corpus
|type= RBID
|clé=
|texte=
}}
| This area was generated with Dilib version V0.6.32. |  |