An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis
Identifieur interne : 000185 ( Pmc/Corpus ); précédent : 000184; suivant : 000186An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis
Auteurs : Loet LeydesdorffSource :
- Scientometrics [ 0138-9130 ] ; 2012.
Abstract
One is inclined to conceptualize impact in terms of citations per publication, and thus as an average. However, citation distributions are skewed, and the average has the disadvantage that the number of publications is used in the denominator. Using hundred percentiles, one can integrate the normalized citation curve and develop an indicator that can be compared across document sets because percentile ranks are defined at the article level. I apply this indicator to the set of 58 journals in the WoS Subject Category of “Nanoscience & nanotechnology,” and rank journals, countries, cities, and institutes using non-parametric statistics. The significance levels of results can thus be indicated. The results are first compared with the ISI-impact factors, but this Integrated Impact Indicator (I3) can be used with any set downloaded from the (
Url:
DOI: 10.1007/s11192-012-0750-5
PubMed: 23293403
PubMed Central: 3535360
Links to Exploration step
PMC:3535360Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis</title><author><name sortKey="Leydesdorff, Loet" sort="Leydesdorff, Loet" uniqKey="Leydesdorff L" first="Loet" last="Leydesdorff">Loet Leydesdorff</name><affiliation><nlm:aff id="Aff1">Amsterdam School of Communications Research (ASCoR), University of Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, The Netherlands</nlm:aff></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">23293403</idno><idno type="pmc">3535360</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3535360</idno><idno type="RBID">PMC:3535360</idno><idno type="doi">10.1007/s11192-012-0750-5</idno><date when="2012">2012</date><idno type="wicri:Area/Pmc/Corpus">000185</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis</title><author><name sortKey="Leydesdorff, Loet" sort="Leydesdorff, Loet" uniqKey="Leydesdorff L" first="Loet" last="Leydesdorff">Loet Leydesdorff</name><affiliation><nlm:aff id="Aff1">Amsterdam School of Communications Research (ASCoR), University of Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, The Netherlands</nlm:aff></affiliation></author></analytic><series><title level="j">Scientometrics</title><idno type="ISSN">0138-9130</idno><idno type="eISSN">1588-2861</idno><imprint><date when="2012">2012</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><p>One is inclined to conceptualize impact in terms of citations per publication, and thus as an average. However, citation distributions are skewed, and the average has the disadvantage that the number of publications is used in the denominator. Using hundred percentiles, one can integrate the normalized citation curve and develop an indicator that can be compared across document sets because percentile ranks are defined at the article level. I apply this indicator to the set of 58 journals in the WoS Subject Category of “Nanoscience & nanotechnology,” and rank journals, countries, cities, and institutes using non-parametric statistics. The significance levels of results can thus be indicated. The results are first compared with the ISI-impact factors, but this Integrated Impact Indicator (I3) can be used with any set downloaded from the (<italic>Social</italic>) <italic>Science Citation Index</italic>. The software is made publicly available at the Internet. Visualization techniques are also specified for evaluation by positioning institutes on Google Map overlays.</p></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Bensman, Sj" uniqKey="Bensman S">SJ Bensman</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bensman, Sj" uniqKey="Bensman S">SJ Bensman</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bonaccorsi, A" uniqKey="Bonaccorsi A">A Bonaccorsi</name></author><author><name sortKey="Vargas, J" uniqKey="Vargas J">J Vargas</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Besselaar, P" uniqKey="Besselaar P">P Besselaar</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author><author><name sortKey="Mutz, R" uniqKey="Mutz R">R Mutz</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author><author><name sortKey="Mutz, R" uniqKey="Mutz R">R Mutz</name></author><author><name sortKey="Neuhaus, C" uniqKey="Neuhaus C">C Neuhaus</name></author><author><name sortKey="Daniel, Hd" uniqKey="Daniel H">HD Daniel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author><author><name sortKey="Schier, H" uniqKey="Schier H">H Schier</name></author><author><name sortKey="Marx, W" uniqKey="Marx W">W Marx</name></author><author><name sortKey="Daniel, Hd" uniqKey="Daniel H">HD Daniel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Boyack, Kw" uniqKey="Boyack K">KW Boyack</name></author><author><name sortKey="Klavans, R" uniqKey="Klavans R">R Klavans</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Chan, Lm" uniqKey="Chan L">LM Chan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Egghe, L" uniqKey="Egghe L">L Egghe</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gingras, Y" uniqKey="Gingras Y">Y Gingras</name></author><author><name sortKey="Lariviere, V" uniqKey="Lariviere V">V Larivière</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gl Nzel, W" uniqKey="Gl Nzel W">W Glänzel</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gl Nzel, W" uniqKey="Gl Nzel W">W Glänzel</name></author><author><name sortKey="Schubert, A" uniqKey="Schubert A">A Schubert</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Gl Nzel, W" uniqKey="Gl Nzel W">W Glänzel</name></author><author><name sortKey="Thijs, B" uniqKey="Thijs B">B Thijs</name></author><author><name sortKey="Schubert, A" uniqKey="Schubert A">A Schubert</name></author><author><name sortKey="Debackere, K" uniqKey="Debackere K">K Debackere</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hirsch, Je" uniqKey="Hirsch J">JE Hirsch</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Huang, C" uniqKey="Huang C">C Huang</name></author><author><name sortKey="Notten, A" uniqKey="Notten A">A Notten</name></author><author><name sortKey="Rasters, N" uniqKey="Rasters N">N Rasters</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kamada, T" uniqKey="Kamada T">T Kamada</name></author><author><name sortKey="Kawai, S" uniqKey="Kawai S">S Kawai</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Levine, G" uniqKey="Levine G">G Levine</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Bornmann, L" uniqKey="Bornmann L">L Bornmann</name></author><author><name sortKey="Mutz, R" uniqKey="Mutz R">R Mutz</name></author><author><name sortKey="Opthof, T" uniqKey="Opthof T">T Opthof</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Opthof, T" uniqKey="Opthof T">T Opthof</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Rafols, I" uniqKey="Rafols I">I Rafols</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author><author><name sortKey="Shin, Jc" uniqKey="Shin J">JC Shin</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Lundberg, J" uniqKey="Lundberg J">J Lundberg</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Moed, Hf" uniqKey="Moed H">HF Moed</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Moed, Hf" uniqKey="Moed H">HF Moed</name></author><author><name sortKey="Bruin, Re" uniqKey="Bruin R">RE Bruin</name></author><author><name sortKey="Leeuwen, Tn" uniqKey="Leeuwen T">TN Leeuwen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Narin, F" uniqKey="Narin F">F Narin</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Nederhof, Aj" uniqKey="Nederhof A">AJ Nederhof</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Opthof, T" uniqKey="Opthof T">T Opthof</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Pudovkin, Ai" uniqKey="Pudovkin A">AI Pudovkin</name></author><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Pudovkin, Ai" uniqKey="Pudovkin A">AI Pudovkin</name></author><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Radicchi, F" uniqKey="Radicchi F">F Radicchi</name></author><author><name sortKey="Castellano, C" uniqKey="Castellano C">C Castellano</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rafols, I" uniqKey="Rafols I">I Rafols</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Rafols, I" uniqKey="Rafols I">I Rafols</name></author><author><name sortKey="Porter, A" uniqKey="Porter A">A Porter</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rousseau, R" uniqKey="Rousseau R">R Rousseau</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Schubert, A" uniqKey="Schubert A">A Schubert</name></author><author><name sortKey="Braun, T" uniqKey="Braun T">T Braun</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Seglen, Po" uniqKey="Seglen P">PO Seglen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Seglen, Po" uniqKey="Seglen P">PO Seglen</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Sheskin, Dj" uniqKey="Sheskin D">DJ Sheskin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Small, H" uniqKey="Small H">H Small</name></author><author><name sortKey="Garfield, E" uniqKey="Garfield E">E Garfield</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Small, H" uniqKey="Small H">H Small</name></author><author><name sortKey="Sweeney, E" uniqKey="Sweeney E">E Sweeney</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vinkler, P" uniqKey="Vinkler P">P Vinkler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vinkler, P" uniqKey="Vinkler P">P Vinkler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vinkler, P" uniqKey="Vinkler P">P Vinkler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Vinkler, P" uniqKey="Vinkler P">P Vinkler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Waltman, L" uniqKey="Waltman L">L Waltman</name></author><author><name sortKey="Eck, Nj" uniqKey="Eck N">NJ Eck</name></author><author><name sortKey="Leeuwen, Tn" uniqKey="Leeuwen T">TN Leeuwen</name></author><author><name sortKey="Visser, Ms" uniqKey="Visser M">MS Visser</name></author><author><name sortKey="Raan, Afj" uniqKey="Raan A">AFJ Raan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Waltman, L" uniqKey="Waltman L">L Waltman</name></author><author><name sortKey="Eck, Nj" uniqKey="Eck N">NJ Eck</name></author><author><name sortKey="Leeuwen, Tn" uniqKey="Leeuwen T">TN Leeuwen</name></author><author><name sortKey="Visser, Ms" uniqKey="Visser M">MS Visser</name></author><author><name sortKey="Raan, Afj" uniqKey="Raan A">AFJ Raan</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zhou, P" uniqKey="Zhou P">P Zhou</name></author><author><name sortKey="Leydesdorff, L" uniqKey="Leydesdorff L">L Leydesdorff</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Zitt, M" uniqKey="Zitt M">M Zitt</name></author><author><name sortKey="Small, H" uniqKey="Small H">H Small</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">Scientometrics</journal-id><journal-id journal-id-type="iso-abbrev">Scientometrics</journal-id><journal-title-group><journal-title>Scientometrics</journal-title></journal-title-group><issn pub-type="ppub">0138-9130</issn><issn pub-type="epub">1588-2861</issn><publisher><publisher-name>Springer Netherlands</publisher-name><publisher-loc>Dordrecht</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">23293403</article-id><article-id pub-id-type="pmc">3535360</article-id><article-id pub-id-type="publisher-id">750</article-id><article-id pub-id-type="doi">10.1007/s11192-012-0750-5</article-id><article-categories><subj-group subj-group-type="heading"><subject>Article</subject></subj-group></article-categories><title-group><article-title>An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis</article-title></title-group><contrib-group><contrib contrib-type="author" corresp="yes"><name><surname>Leydesdorff</surname><given-names>Loet</given-names></name><address><email>loet@leydesdorff.net</email><uri>http://www.leydesdorff.net</uri></address><xref ref-type="aff" rid="Aff1"></xref></contrib><aff id="Aff1">Amsterdam School of Communications Research (ASCoR), University of Amsterdam, Kloveniersburgwal 48, 1012 CX Amsterdam, The Netherlands</aff></contrib-group><pub-date pub-type="epub"><day>8</day><month>5</month><year>2012</year></pub-date><pub-date pub-type="pmc-release"><day>8</day><month>5</month><year>2012</year></pub-date><pub-date pub-type="ppub"><month>1</month><year>2013</year></pub-date><volume>94</volume><issue>1</issue><fpage>35</fpage><lpage>55</lpage><history><date date-type="received"><day>3</day><month>1</month><year>2012</year></date></history><permissions><copyright-statement>© The Author(s) 2012</copyright-statement></permissions><abstract id="Abs1"><p>One is inclined to conceptualize impact in terms of citations per publication, and thus as an average. However, citation distributions are skewed, and the average has the disadvantage that the number of publications is used in the denominator. Using hundred percentiles, one can integrate the normalized citation curve and develop an indicator that can be compared across document sets because percentile ranks are defined at the article level. I apply this indicator to the set of 58 journals in the WoS Subject Category of “Nanoscience & nanotechnology,” and rank journals, countries, cities, and institutes using non-parametric statistics. The significance levels of results can thus be indicated. The results are first compared with the ISI-impact factors, but this Integrated Impact Indicator (I3) can be used with any set downloaded from the (<italic>Social</italic>) <italic>Science Citation Index</italic>. The software is made publicly available at the Internet. Visualization techniques are also specified for evaluation by positioning institutes on Google Map overlays.</p></abstract><kwd-group xml:lang="en"><title>Keywords</title><kwd>Citation</kwd><kwd>Impact</kwd><kwd>Evaluation</kwd><kwd>Nanotechnology</kwd><kwd>Statistics</kwd><kwd>Standards</kwd></kwd-group><custom-meta-group><custom-meta><meta-name>issue-copyright-statement</meta-name><meta-value>© Akadémiai Kiadó, Budapest, Hungary 2013</meta-value></custom-meta></custom-meta-group></article-meta></front><body><sec id="Sec1" sec-type="introduction"><title>Introduction</title><p>In recent decades, scientometric indicators have increasingly been used for evaluative purposes both in public-policy domains and, for example, in hiring and promoting faculty. Policy makers and research managers need to rely on the quality of these indicators. Recent years, however, have witnessed fierce debate about the use of statistics and standards in evaluation research (Gingras and Larivière <xref ref-type="bibr" rid="CR16">2011</xref>). A transition from parametric statistics (using averages) towards non-parametric statistics (using percentiles) provides advantages, but implies a different conceptualization of “impact.”</p><p>Non-parametric statistics enable us to honour both productivity and quality, whereas the impact may be lower in the case of averaging for the sole reason of a higher productivity. These statistics share this appreciation of both productivity and citation rates with the <italic>h</italic>-index (Hirsch <xref ref-type="bibr" rid="CR20">2005</xref>), but they differ from the <italic>h</italic>-index in that a range of tests for the significance of the impact (above or below expectation) becomes available. Less-cited papers can thus be appreciated proportionally, while the <italic>h</italic>-index uses the <italic>h</italic>-value as a threshold for the cutoff of the tails of the distributions (cf. Glänzel <xref ref-type="bibr" rid="CR17">2007</xref>; Vinkler <xref ref-type="bibr" rid="CR63">2010</xref>, <xref ref-type="bibr" rid="CR64">2011</xref>).</p><p>Scientometric indicators first require normalization because of differences in publication and citation practices among fields of science. For example, impact factors (IFs) in psychology are on average larger than in sociology by an order of magnitude (Leydesdorff <xref ref-type="bibr" rid="CR26">2008</xref>). Mathematicians provide short reference lists (on the order of five references), while life-scientists often provide more than 40 references. The chances of being highly-cited and positively evaluated would thus vary among the sciences for statistical reasons (Garfield <xref ref-type="bibr" rid="CR14">1979a</xref>).</p><p>Two conventions have been devised in the past for normalization: (1) normalization in terms of fields of science, and (2) comparison of the citation rate with the world average. For the delineation of fields of science, scientometricians often turn to journal classification schemes provided by the database producer—such as the WoS Subject Categories of the (<italic>Social</italic>) <italic>Science Citation Index</italic> (SCI and SSCI).<xref ref-type="fn" rid="Fn1">1</xref> Journals, however, can be assigned to more than a single category, and this categorization is not “literary warranted”; that is, it is not updated systematically to follow the development of the scientific literature (Bensman and Leydesdorff <xref ref-type="bibr" rid="CR3">2009</xref>; Chan <xref ref-type="bibr" rid="CR11">1999</xref>).</p><p>The sources of error in these field delineations are difficult to control using journals as units of analysis because it is questionable whether journals themselves are mono-disciplinary units in terms of their content (Boyack and Klavans <xref ref-type="bibr" rid="CR10">2011</xref>; Pudovkin and Garfield <xref ref-type="bibr" rid="CR46">2002</xref>; Rafols and Leydesdorff <xref ref-type="bibr" rid="CR49">2009</xref>). Furthermore, one can expect the institutional units under study—such as universities—to be organized disciplinarily and interdisciplinarily, but from a different perspective. Therefore, refinement of the journal categories cannot solve the problem of field delineation (Leydesdorff <xref ref-type="bibr" rid="CR25">2006</xref>; Rafols et al. in press). Perhaps, paper-based classifications such as the Medical Subject Headings (MeSH) of the Medline database would be more appropriate, but the complexity of the handling of citations in relation to more than a single database has hitherto been technically difficult (Bornmann et al. <xref ref-type="bibr" rid="CR8">2008</xref>, <xref ref-type="bibr" rid="CR9">2011</xref>).</p><p>The second problem—that is, the comparison to a world average—was addressed early in the scientometric enterprise by Schubert and Braun (<xref ref-type="bibr" rid="CR54">1986</xref>), who proposed comparing the mean observed citation rate (MOCR) with the corresponding mean <italic>expected</italic> citation rate (MECR) as the average of papers of the same datatype and publication year within a reference set (representing, for example, a field of science). The Relative Citation Rate (RCR = MOCR/MECR) is thus normalized with unity as the world average.</p><p>The two normalizations—in terms of journals and/or fields, and with reference to a world average—can be combined. The Leiden “crown indicator” (CPP/FCSm), for example, was defined by Moed et al. (<xref ref-type="bibr" rid="CR39">1995</xref>) as the average citation rate of the sample under study (citations per publication: CPP) divided by the mean citation score at the field level (FCSm), formalized as follows<xref ref-type="fn" rid="Fn2">2</xref>:<disp-formula id="Equ1"><label>1</label><alternatives><tex-math id="M1">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ {\text{CPP}}/{\text{FCSm}} = \frac{{\sum\nolimits_{i = 1}^{m} {{\text{observed}}_{i} /m} }}{{\sum\nolimits_{i = 1}^{n} {{\text{expected}}_{i} } /n}} $$\end{document}</tex-math><graphic xlink:href="11192_2012_750_Article_Equ1.gif" position="anchor"></graphic></alternatives></disp-formula></p><p>Similarly, the current evaluation standard of the ECOOM center in Louvain (Belgium) uses the Normalized Mean Citation Rate (NMCR; Glänzel et al. <xref ref-type="bibr" rid="CR19">2009</xref>) based on the same principles, but sometimes using their own classification system of journals for the field delineation (Glänzel and Schubert <xref ref-type="bibr" rid="CR18">2003</xref>; cf. Rafols and Leydesdorff <xref ref-type="bibr" rid="CR49">2009</xref>).</p><p>The division of two means results in mathematical inconsistency because the order of operations is violated: according to this rule, one should first divide and then add. Instead, one could normalize as follows:<disp-formula id="Equ2"><label>2</label><alternatives><tex-math id="M2">\documentclass[12pt]{minimal}
\usepackage{amsmath}
\usepackage{wasysym}
\usepackage{amsfonts}
\usepackage{amssymb}
\usepackage{amsbsy}
\usepackage{mathrsfs}
\usepackage{upgreek}
\setlength{\oddsidemargin}{-69pt}
\begin{document}$$ {\text{MNCS}} = {{\left( {\sum\nolimits_{i = 1}^{n} {\frac{{{\text{observed}}_{i} }}{{{\text{expected}}_{i} }}} } \right)} \mathord{\left/ {\vphantom {{\left( {\sum\nolimits_{i = 1}^{n} {\frac{{{\text{observed}}_{i} }}{{{\text{expected}}_{i} }}} } \right)} n}} \right. \kern-\nulldelimiterspace} n} $$\end{document}</tex-math><graphic xlink:href="11192_2012_750_Article_Equ2.gif" position="anchor"></graphic></alternatives></disp-formula></p><p>In reaction to this critique by Opthof and Leydesdorff (<xref ref-type="bibr" rid="CR43">2010</xref>; cf. Lundberg <xref ref-type="bibr" rid="CR37">2007</xref>), the Centre for Science and Technology Studies (CWTS) in Leiden has changed the crown indicator (Waltman et al. <xref ref-type="bibr" rid="CR66">2011a</xref>), but not all other centers have followed suit. CWTS called this “new crown indicator” the Mean Normalized Citation Score or MNCS. One advantage of this indicator is that the mean is a statistics with a standard deviation, and consequently a standard error for the measurement can be defined. Waltman et al. (<xref ref-type="bibr" rid="CR67">2011b</xref>) have shown that this new indicator is mathematically consistent (cf. Egghe <xref ref-type="bibr" rid="CR12">2012</xref>; Vinkler <xref ref-type="bibr" rid="CR65">2012</xref>).</p><p>In the sideline of these debates about citation indicators, two further steps were taken. First, Leydesdorff and Opthof (<xref ref-type="bibr" rid="CR33">2010</xref>) proposed abandoning the idea of journal classification, instead using the <italic>citing</italic> articles as the reference set of the <italic>cited</italic> articles, and then normalizing by counting each citation in proportion to the length of the reference list in the citing paper (1/<italic>NRef</italic>; the field <italic>NRef</italic> is available in the <italic>Science Citation Index</italic>). Differences in citation behaviour among fields of science can thus be normalized at the source. This fractional counting of citations was previously used by Small and Sweeney (<xref ref-type="bibr" rid="CR60">1985</xref>) for the mapping of co-citations; and “source-normalization” has been proposed by Zitt and Small (<xref ref-type="bibr" rid="CR69">2008</xref>) and Moed (<xref ref-type="bibr" rid="CR38">2010</xref>), albeit in other contexts.</p><p>Leydesdorff and Bornmann (<xref ref-type="bibr" rid="CR28">2011a</xref>) could show that the differences in IFs among fields were no longer statistically significant when using such fractional counting instead of counting each citation as a full point. Another advantage of this methodology is the possibility to evaluate interdisciplinary units fractionally across different disciplines. Zhou and Leydesdorff (<xref ref-type="bibr" rid="CR68">2011</xref>), for example, have shown that the department of Chinese Language and Literature of the Tsinghua University in Beijing was upgraded in the evaluation from 19th to 2nd position by using fractional counting (given the scarcity of journal citations in the humanities; Leydesdorff et al. <xref ref-type="bibr" rid="CR31">2011a</xref>; Nederhof <xref ref-type="bibr" rid="CR42">2006</xref>). In an evaluation of interdisciplinary programs in innovation studies, Rafols et al. (in press) found fractional counting of the citations to improve the measurement greatly, particularly if the set was limited to documents with more than ten references (cf. Leydesdorff <xref ref-type="bibr" rid="CR27">2012</xref>; Leydesdorff et al. in preparation).</p><p>In a second contribution to this controversy, Bornmann and Mutz (<xref ref-type="bibr" rid="CR7">2011</xref>) proposed using percentile ranks instead of averages (Seglen <xref ref-type="bibr" rid="CR55">1992</xref>, <xref ref-type="bibr" rid="CR56">1997</xref>), in accordance with the standard practice of the <italic>Science & Engineering Indicators</italic> of the US National Science Board (NSB, <xref ref-type="bibr" rid="CR41">2010</xref>). Six percentile ranks are distinguished: the top-1 %, top-5 %, top-10 %, top-25 %, top-50 %, and bottom-50 %. If a one is assigned to a paper in the bottom category, and a six to a paper in the top-1 %, it can be shown that the random expectation is a score of 1.91.</p><p>The teams of Bornmann & Mutz and Leydesdorff & Opthof thereafter joined forces and evaluated the seven principal investigator (PIs) of the set that originally triggered this debate (Opthof and Leydesdorff <xref ref-type="bibr" rid="CR43">2010</xref>), but using percentile ranks, and then compared the results with those obtained by using the full range of hundred percentiles. In this study, they proposed one further step: one should distinguish evaluation of the seven document sets (of publications) as independent samples—which is the current practice—from their evaluation as subsamples of a reference set.</p><p>Among the seven PIs, for example, the one ranked highest had 23 papers of which three were in the top-1 %. Each of these three papers contributes (1/23) * 6 = 0.261 points to the total score of this PI (PI1). However, PI6 had 65 papers; a single paper in the top-1 % would in this case add (1/65) * 6 = 0.092 points to his/her total score. In other words, one should no longer consider these samples as independent, but instead use the grand sum of the total set (<italic>N</italic> = 248) so that each PI is credited equally with (1/248) * 6 = 0.024 points for each paper in the top-1 % category. By using this measure, PI6 was upgraded to first position, and PI1 downgraded to fifth in the ranking (Leydesdorff et al. <xref ref-type="bibr" rid="CR32">2011b</xref>, Table 1, at p. 1375).</p><p>The reference set thus provides a standard, and each subset is evaluated as a part of this superset. This reference set can be any encompassing superset, including, for example, a definition of the field in terms of relevant journals or specific keywords (e.g., Bonaccorsi and Vargas <xref ref-type="bibr" rid="CR4">2010</xref>). The crucial point is that each paper adds to the impact score proportionally, that is, according to its percentile rank. Impact is thus no longer dependent on dividing by the number of papers in a specific subset: if two researchers have an equal number of papers in the highest category, adding more papers to one of the sets will add to its percentage impact even if the added papers are in a lower category. In the case of averaging (or using the median) these additional papers would decrease the impact, and thus “punish” productivity.</p><p>Leydesdorff and Bornmann (<xref ref-type="bibr" rid="CR29">2011b</xref>) have elaborated this approach into an indicator: the <italic>Integrated Impact Indicator</italic> (I3). In the meantime, a website is available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/software/i3">http://www.leydesdorff.net/software/i3</ext-link> which provides the relevant routines. The indicator can formally be written as follows: I3 = ∑<sub><italic>i</italic></sub><italic>x</italic><sub><italic>i</italic></sub> * <italic>n</italic>(<italic>x</italic><sub><italic>i</italic></sub>), in which <italic>x</italic><sub><italic>i</italic></sub> denotes the percentile (rank) value <italic>i</italic>, and <italic>n</italic> the number of papers with this value. I3 leaves the parametric domain of averages behind and moves to non-parametric statistics using percentiles. The percentiles can be computed as a continuous random variablze. Aggregation of the values in the six percentile rank classes distinguished above, leads to an indicator (PR6) that weights highly-cited papers more than lower-cited ones. I shall use both these indicators throughout this study.</p><p>Using I3 or PR6, one can test whether a contribution is statistically significant above or below expectation using the <italic>z</italic>-test, and one can also compare citation distributions in terms of significant differences using Dunn’s test. These are two distinct questions which should not be confused. For example, the citation distribution of the <italic>Proceedings of the National Academy of Science of the USA</italic> and the <italic>Proceedings of the Estonian Academy of Science</italic> are not significantly different, but the impact of the former is significantly above the expected impact using I3, while this is not the case for the latter (Leydesdorff and Bornmann <xref ref-type="bibr" rid="CR28">2011a</xref>).</p><p>The concept of impact has thus been redefined. Using averages (as in the case of IFs), the <italic>N</italic> in the denominator has an adverse effect on productivity. When a researcher coauthors with his PhD students, the average citation rate of the group is likely to go down, but the total impact of this research project increases. As Bensman (<xref ref-type="bibr" rid="CR2">2007</xref>) noted, Garfield (<xref ref-type="bibr" rid="CR13">1972</xref>, <xref ref-type="bibr" rid="CR15">1979b</xref>) chose to normalize the ISI-impact factor by dividing by <italic>N</italic> in order to prevent the larger journals from overshadowing the smaller. Bensman then proposed using “Total Citations” instead as an indicator of reputation, because the latter correlated more clearly with the subjective appreciation of faculty than the IFs. Our percentile-rank approach, however, appreciates that highly-cited papers should be weighted more than less frequently cited ones.</p><p>The division by <italic>N</italic> was just too coarse: using an average assumes a normal distribution, whereas the skewness in different citation distributions can be made comparable when normalized on the 1–100 scale of percentiles. How the percentiles are aggregated, for example, into the six percentile ranks of the NSF used by Bornmann and Mutz (<xref ref-type="bibr" rid="CR7">2011</xref>), or in terms of quartiles, is a normative decision which can be made independently from the technical analysis in terms of hundred percentiles. In what follows, I use both the scheme of six percentile ranks and the baseline of 100 percentiles to evaluate the case of journals subsumed under the WoS Subject Category of “Nanoscience & nanotechnology.” In order to make my results comparable with the IFs, I use whole counts and 2 years of cited data (2007 and 2008) and compare with the IF-2009 (the most recent one available at the time of this research; Spring 2011). As noted, I3 is defined more abstractly; it can be used with any reference set and also with fractional counts.</p></sec><sec id="Sec2"><title>The “nano-set” in the <italic>Science Citation Index</italic></title><p>The WoS Subject Category “Nanoscience & nanotechnology” (NS) entered the SCI in 2005 with 27 journals and more than doubled to include 59 journals by 2009. One of these journals (<italic>ACM Applied Materials & Interfaces</italic>) was added to the database only in 2009, and consequently no records for the two previous years were included (as of the time of this study). I limit the set to the 31,644 citable items in these 58 journals during the years 2007 and 2008 (that is: 25,271 articles, 5,488 proceedings papers, 709 reviews, and 176 letters). The percentile of each paper is determined with reference to the set with the same publication year and document type, respectively.</p><p>A simple counting rule for percentiles is the number of papers with lower citation rates divided by the total number of similar records in the set. The resulting value can be rounded into percentiles or used as a continuous random variable (so-called “quantiles”). The advantage of this counting rule over other possible ones is that tied ranks are accorded their highest possible values. Other rules are also possible: Pudovkin and Garfield (<xref ref-type="bibr" rid="CR47">2009</xref>), for example, first averaged tied ranks.</p><p>Before turning to the methodological details, let me first specify the problem by taking the journal with the highest IF in this category—that is, <italic>Nature Nanotechnology</italic> with IF-2009 = 26,309—and comparing it with the third in rank: <italic>Nano Letters</italic> with IF-2009 = 9.991.<xref ref-type="fn" rid="Fn3">3</xref> Figure <xref rid="Fig1" ref-type="fig">1</xref> shows that the lower IF of <italic>Nano Letters</italic> is entirely due to the large tail of the distribution. In the left part of this figure (the 199 most-highly cited papers), the average citation score of <italic>Nano Letters</italic> (89.78 ± 0.27) is higher than that of <italic>Nature Nanotechnology</italic> (66.45 ± 0.38)<italic>.</italic><fig id="Fig1"><label>Fig. 1</label><caption><p>Citation curves for <italic>Nature Nanotechnology</italic> (<italic>N</italic> = 199 publications) and <italic>Nano Letters</italic> (<italic>N</italic> = 1,506)</p></caption><graphic xlink:href="11192_2012_750_Fig1_HTML" id="MO3"></graphic></fig></p><p>Analogously, using an average value would underestimate productivity of a university group because of the <italic>N</italic> in the denominator. If one adds two groups together or merges these two journals (in a thought experiment), the impact of the merged group should, in my opinion, be not the average of the two previous groups, but their sumtotal. However, this summation needs to be qualified: papers in the top-1 % range are to be added to other papers in the top-1 %, etc. <italic>mutatis mutandis</italic>. If we compare, thus qualified, the six percentile ranks for these two journals, the results can be seen in Fig. <xref rid="Fig2" ref-type="fig">2</xref>.<fig id="Fig2"><label>Fig. 2</label><caption><p>Distribution of six percentile ranks of publications in <italic>Nano Letters</italic> and <italic>Nature Nanotechnology</italic> with reference to the 58 journals of the ISI Subject Category “Nanoscience & nanotechnology”</p></caption><graphic xlink:href="11192_2012_750_Fig2_HTML" id="MO4"></graphic></fig></p><p>Figure <xref rid="Fig2" ref-type="fig">2</xref> shows that <italic>Nano Letters</italic> outperforms <italic>Nature Nanotechnology</italic> in all six classes. However, if one divides the total scores by the number of publications in each category, <italic>Nano Letters</italic> becomes the less important journal (in all categories). Thus one should not divide, but integrate: impacts adds up. The impact of two collisions is the (vector) sum of the two momenta. The use of the word “impact factor” to denote an average value has confused the semantics for decades (Sher and Garfield <xref ref-type="bibr" rid="CR57">1965</xref>; Garfield <xref ref-type="bibr" rid="CR13">1972</xref>).</p><p>The integrals in this stepwise function are equal to ∑<sub><italic>i</italic></sub><italic>x</italic><sub><italic>i</italic></sub> * <italic>f</italic>(<italic>x</italic><sub><italic>i</italic></sub>) in which <italic>x</italic> represents the percentile rank and <italic>f</italic>(<italic>x</italic>) the frequency of that rank. For example, <italic>i</italic> = 6 in the case above, or <italic>i</italic> = 100 when using all percentiles as classes. The hundred percentiles can be considered as a continuous random variable, and these “quantiles” form the baseline. Note that the percentile is thereafter a characteristic of each individual paper. Thus, different aggregations of papers are possible, for example in terms of journals, nations, institutes, cities, or authors. The function integrates both the number of papers (the “mass”) and their respective quality in terms of being-cited, but after normalization represents them as percentiles with reference to a set.</p><p>Figure <xref rid="Fig3" ref-type="fig">3</xref> shows the results of a factor analysis using the 58 journals attributed to the WoS Subject Category “Nanoscience & nanotechnology” as cases, and the numbers of publications, citations, IFs, I3 values for both 100 and for 6 classes (PR6). The citations and publications correlate highly (<italic>r</italic> = 0.835, <italic>p</italic> ≤ 0.01) because citations correlate with size (Bensman <xref ref-type="bibr" rid="CR2">2007</xref>). In my opinion, an impact indicator can only be meaningful if it also correlates with these indicators of productivity and quality. But this is not the case for the IF, because it is based on dividing these two indicators.<fig id="Fig3"><label>Fig. 3</label><caption><p>Factor analysis (varimax rotated) of the number of publications, citations, IFs, I3-values (using 100 percentiles), and values using 6 percentile ranks (PR6); the 58 journals attributed to the ISI Subject Category “Nanoscience & nanotechnology” are the cases</p></caption><graphic xlink:href="11192_2012_750_Fig3_HTML" id="MO5"></graphic></fig></p><p>I3 correlates with both citations (<italic>r</italic> = 0.935, <italic>p</italic> ≤ 0.01) and publications (<italic>r</italic> = 0.963, <italic>p</italic> ≤ 0.01) more than these two values correlate mutually: this then takes both productivity and quality into account. PR6 is slightly more sensitive to high citation rates than I3 (based on hundred percentiles), but the difference between the two indicators is not significant.</p></sec><sec id="Sec3" sec-type="materials|methods"><title>Methods and materials</title><p>The 58 journals in the ISI Subject Category “Nanoscience & nanotechnology” conveniently contain a large sample: 31,644 citable items, with 65,173 addresses, and 146,432 authors. All records were downloaded on February 25, 2011. Dedicated software was developed and brought online at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/software/i3">http://www.leydesdorff.net/software/i3</ext-link>. The routines add the values for the two relevant indicators (I3 and PR6) at the article level to a set downloaded from the Web-of-Science (WoS) and organized into relational databases. The percentiles can be summed into categories using the various possible schemes for an evaluation. As noted, I use the six ranks of the NSF as an example in addition to the quantiles (between 0 and 100) as a continuous random variable.</p><p>The simple counting rule for quantiles—specified above—may generate problems when a reference set is smaller than 100. For example, if a journal includes among its publications only 10 reviews each year, the highest possible percentile would be the 90 %—nine out of ten—whereas this could be the 99th percentile. Rüdiger Mutz (personal communication, February 18, 2011) suggested adding 0.9 to each percentile value, which solved part of the problem (Leydesdorff and Bornmann <xref ref-type="bibr" rid="CR29">2011b</xref>). Rousseau (<xref ref-type="bibr" rid="CR52">2012</xref>) proposed adopting as a counting rule not “less than” (<), but “less than or equal to” (≤) the item under study. All papers would then have a chance to be in the top rank of the 100th percentile, but the resulting values are higher. I did not apply this solution in the current study (Leydesdorff and Bornmann in press). Most recently, Schreiber (in press) proposed another solution to this problem.</p><p>The file of 31,644 citable items each with a value for “times cited” (“TC” provided by the database), I3 (for the hundred percentiles), and PR6 (for the six classes of the NSF) can be imported into SPSS (v.18); and then the routine “Compare Means,” using the journals—countries, cities, etc., respectively—in each set as the independent (grouping) variable, can be used for determining all relevant statistics. (This routine also allows for bootstrapping and the determination of confidence levels.) The I3-values and PR6-values are based on summation for the units of analysis under evaluation. The standard error of measurement (SEM) values are based on averaging and will not be used below, but are also available from this routine.</p><p>The I3 and PR6 values can be recomposed by aggregating for each subset—for example, each journal or a subset with a specific institutional address—and expressed as a percentage of the total I3 (or PR6) value for the set. The routine isi2i3.exe also provides tables at the level of authors, institutes,<xref ref-type="fn" rid="Fn4">4</xref> countries, and journals, with aggregates of these variables. (Alternatively, one can use the procedure “Aggregate cases” in SPSS or generate pivot tables in Excel.) The absolute numbers can be expressed as percentages of the set which can then be compared and used for ranking. Examples will be provided in the results section below.</p><p>Both I3 and PR6 are size-dependent because a unit with 1,000 publications has 10 times more chance to have one in the top-1 % range than a unit with only 100 publications. These indicator values can be tested against the publication rates: <italic>ex ante</italic>, all publications have an equal chance to be cited. One can use the <italic>z</italic>-test for two independent proportions to test observed versus expected rates for their statistical significance (Sheskin <xref ref-type="bibr" rid="CR58">2011</xref>, p. 656).<xref ref-type="fn" rid="Fn5">5</xref> Note that the impact I3 is not independent of the number of publications. I shall return to this issue below.</p><p>Citation curves (such as in Fig. <xref rid="Fig1" ref-type="fig">1</xref>) can be tested against each other using “Multiple Comparisons” with Bonferroni correction in SPSS. When the confidence levels do not intersect, the distributions are flagged as significantly different. In the case of non-parametric statistics, Dunn’s test can be simulated by using least significant differences (LSD) with family-wise correction for Type-I error probability (Levine <xref ref-type="bibr" rid="CR23">1991</xref>, at pp. 68 ff.).<xref ref-type="fn" rid="Fn6">6</xref> The routine in SPSS is limited to a maximum of 50 comparisons. Alternatively, one can use the Mann–Whitney <italic>U</italic> test with the same Bonferroni correction between each individual pair.</p><p>Journals (etc.) with significantly different or similar citation patterns, can be visualized as a graph in which homogenous sets are connected while significantly different nodes are not. I shall use the algorithm of Kamada and Kawai (<xref ref-type="bibr" rid="CR22">1989</xref>) in Pajek for this visualization. The <italic>k</italic>-core set which are most homogenous in terms of citation distributions can thus be visualized. Analogously, other variables which are attributed at the article level, such as the percentile values I3 and PR6, can be analyzed. Differences and similarities in citation distributions can be expected to change after normalization in terms of percentile values. The latter exhibit differences in impact (I3 and/or PR6).</p><p>In summary, one can assess each percentage impact (with reference to a set) in terms of whether the contribution (expressed as a percentage impact of the set) is significantly above or below the expectation. In accordance with the convention in SPSS but using plus and minus signs, I use double <sup>++</sup> for above and <sup>− −</sup> for below expectation at the 1 % level of significance testing, and single <sup>+</sup> and <sup>−</sup> at the 5 % level. The significance of the <italic>contribution</italic> is a different question from whether its ranking is based on significant <italic>differences</italic> in the underlying distributions of the citations or I3-values. The latter analysis enables us to indicate groups which may differ in size but otherwise be homogenous.</p></sec><sec id="Sec4" sec-type="results"><title>Results</title><sec id="Sec5"><title>Journal evaluation</title><p>Table <xref rid="Tab1" ref-type="table">1</xref> provides the rankings of the 20 nano-journals most highly ranked in terms of their impact measured as I3 (column b). As expected, the relation with the IFs is virtually absent (<italic>r</italic> = 0.178; n.s.). <italic>Nano Letters</italic> ranks second on this list in terms of its IF, with <italic>Nature Nanotechnology</italic> in first place. But while <italic>Nano Letters</italic> ranks third from the top in terms of I3, <italic>Nature Nanotechnology</italic> in this case occupies only the 19th position.<table-wrap id="Tab1"><label>Table 1</label><caption><p>Rankings between 20 journals of “Nanoscience & nanotechnology” with highest values on I3 (expressed as percentages of the sum) compared with IFs, total citations, and % I3 (PR6)</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left">Journal</th><th align="left">No. of papers<break></break>(a)</th><th align="left">% I3<break></break>(b)</th><th align="left">IF 2009<break></break>(c)</th><th align="left">Total citations<break></break>(d)</th><th align="left">% I3 (PR6)<break></break>(e)</th></tr></thead><tbody><tr><td align="left">1</td><td align="left"><italic>J Phys Chem C</italic></td><td char=" " align="char">5,585 [1]</td><td char=" " align="char">20.71 [1]<sup>++</sup></td><td char=" " align="char">4.224 [9]</td><td char="." align="char">62,072</td><td char=" " align="char">18.79 [1]<sup>++</sup></td></tr><tr><td align="left">2</td><td align="left"><italic>Mater Sci Eng A</italic>-<italic>Struct Mater</italic></td><td char=" " align="char">3,222 [2]</td><td char=" " align="char">8.82 [2]<sup>− −</sup></td><td char=" " align="char">1.901 [12]</td><td char="." align="char">15,920</td><td char=" " align="char">8.64 [2]<sup>− −</sup></td></tr><tr><td align="left">3</td><td align="left"><bold><italic>Nano Lett</italic></bold></td><td char=" " align="char">1,507 [6]</td><td char=" " align="char">7.99 [3]<sup>++</sup></td><td char=" " align="char">9.991 [2]</td><td char="." align="char">42,159</td><td char=" " align="char">8.38 [3]<sup>++</sup></td></tr><tr><td align="left">4</td><td align="left"><italic>Nanotechnol</italic></td><td char=" " align="char">2,656 [3]</td><td char=" " align="char">7.96 [4]<sup>− −</sup></td><td char=" " align="char">3.137 [10]</td><td char="." align="char">18,748</td><td char=" " align="char">7.22 [5]<sup>− −</sup></td></tr><tr><td align="left">5</td><td align="left"><italic>Advan Mater</italic></td><td char=" " align="char">1,512 [5]</td><td char=" " align="char">7.61 [5]<sup>++</sup></td><td char=" " align="char">8.379 [3]</td><td char="." align="char">34,581</td><td char=" " align="char">7.59 [4]<sup>++</sup></td></tr><tr><td align="left">6</td><td align="left"><italic>Adv Funct Mater</italic></td><td char=" " align="char">883</td><td char=" " align="char">4.24 [6]<sup>++</sup></td><td char=" " align="char">6.990 [5]</td><td char="." align="char">16,629</td><td char=" " align="char">4.04 [7]<sup>++</sup></td></tr><tr><td align="left">7</td><td align="left"><italic>J Nanosci Nanotechnol</italic></td><td char=" " align="char">1,699 [4]</td><td char=" " align="char">3.74 [7]<sup>− −</sup></td><td char=" " align="char">1.435 [15]</td><td char="." align="char">5,946</td><td char=" " align="char">4.07 [6]<sup>− −</sup></td></tr><tr><td align="left">8</td><td align="left"><italic>Biosens Bioelectron</italic></td><td char=" " align="char">821</td><td char=" " align="char">3.57 [8]<sup>++</sup></td><td char=" " align="char">5.429 [8]</td><td char="." align="char">11,583</td><td char=" " align="char">3.34 [8]<sup>++</sup></td></tr><tr><td align="left">9</td><td align="left"><italic>Scripta Mater</italic></td><td char=" " align="char">1,153</td><td char=" " align="char">3.43 [9]<sup>− −</sup></td><td char=" " align="char">2.949 [11]</td><td char="." align="char">8,656</td><td char=" " align="char">3.13 [11]<sup>− −</sup></td></tr><tr><td align="left">10</td><td align="left"><italic>Microporous Mesoporous Mat</italic></td><td char=" " align="char">1,041</td><td char=" " align="char">3.40 [10]<sup>++</sup></td><td char=" " align="char">2.652 [12]</td><td char="." align="char">7,140</td><td char=" " align="char">3.14 [10]</td></tr><tr><td align="left">11</td><td align="left"><italic>Microelectron Eng</italic></td><td char=" " align="char">1,064</td><td char=" " align="char">3.14 [11]<sup>− −</sup></td><td char=" " align="char">1.488 [13]</td><td char="." align="char">4,090</td><td char=" " align="char">3.17 [9]<sup>−</sup></td></tr><tr><td align="left">12</td><td align="left"><italic>Small</italic></td><td char=" " align="char">595</td><td char=" " align="char">2.58 [12]<sup>++</sup></td><td char=" " align="char">6.171 [7]</td><td char="." align="char">9,811</td><td char=" " align="char">2.40 [14]<sup>++</sup></td></tr><tr><td align="left">13</td><td align="left"><italic>Lab Chip</italic></td><td char=" " align="char">520</td><td char=" " align="char">2.34 [13]<sup>++</sup></td><td char=" " align="char">6.342 [6]</td><td char="." align="char">8,930</td><td char=" " align="char">2.21 [15]<sup>++</sup></td></tr><tr><td align="left">14</td><td align="left"><italic>J Vac Sci Technol B</italic></td><td char=" " align="char">962</td><td char=" " align="char">2.29 [14]<sup>− −</sup></td><td char=" " align="char">1.460 [14]</td><td char="." align="char">3,471</td><td char=" " align="char">2.43 [13]<sup>− −</sup></td></tr><tr><td align="left">15</td><td align="left"><italic>Physica E</italic></td><td char=" " align="char">1,004</td><td char=" " align="char">2.26 [15]<sup>− −</sup></td><td char=" " align="char">1.177 [16]</td><td char="." align="char">3,126</td><td char=" " align="char">2.64 [12]<sup>− −</sup></td></tr><tr><td align="left">16</td><td align="left"><italic>J Micromechanic Microengineer</italic></td><td char=" " align="char">778</td><td char=" " align="char">1.86 [16]<sup>− −</sup></td><td char=" " align="char">0.541 [19]</td><td char="." align="char">186</td><td char=" " align="char">1.86 [16]<sup>− −</sup></td></tr><tr><td align="left">17</td><td align="left"><italic>Acs Nano</italic></td><td char=" " align="char">349</td><td char=" " align="char">1.73 [17]<sup>++</sup></td><td char=" " align="char">7.493 [4]</td><td char="." align="char">6,845</td><td char=" " align="char">1.72 [17]<sup>++</sup></td></tr><tr><td align="left">18</td><td align="left"><italic>Microelectron Rel</italic></td><td char=" " align="char">627</td><td char=" " align="char">1.20 [18]<sup>− −</sup></td><td char=" " align="char">1.117 [17]</td><td char="." align="char">1,767</td><td char=" " align="char">1.44 [19]<sup>− −</sup></td></tr><tr><td align="left">19</td><td align="left"><bold><italic>Nat Nanotechnol</italic></bold></td><td char=" " align="char">199</td><td char=" " align="char">1.20 [19]<sup>++</sup></td><td char=" " align="char">26.309 [1]</td><td char="." align="char">13,224</td><td char=" " align="char">1.47 [18]<sup>++</sup></td></tr><tr><td align="left">20</td><td align="left"><italic>Microsyst Technol</italic></td><td char=" " align="char">475</td><td char=" " align="char">1.00 [20]<sup>− −</sup></td><td char=" " align="char">1.025 [18]</td><td char="." align="char">1,265</td><td char=" " align="char">1.17 [20]<sup>− −</sup></td></tr></tbody></table><table-wrap-foot><p><sup>++</sup><italic>p</italic> < 0.01 above the expectation; <sup>+</sup> <italic>p</italic> < 0.05 above the expectation; <sup>− −</sup> <italic>p</italic> < 0.01 below the expectation; <sup>−</sup> <italic>p</italic> < 0.05 below the expectation</p><p>Ranks are added between brackets</p></table-wrap-foot></table-wrap></p><p>Note that <italic>Nano Letters</italic> ranks only sixth in terms of the number of publications (column a), but third in terms of the impact I3 and PR6 (columns b and e, respectively). Its impact is thus (significantly!) above expectation. <italic>Mater Sci Eng A</italic>-<italic>Struct Mater</italic>,<xref ref-type="fn" rid="Fn7">7</xref> in second position, is larger in terms of numbers of publications, but relatively low in terms of citations. Therefore, it scores significantly below the expected citation rates in columns (b) and (e). Analogously, the <italic>c</italic>/<italic>p</italic> ratio for <italic>Nature Nanotechnology</italic> (13,244/199 =) 66.45 corresponds to its high IF, but, as noted, this is entirely due to the low number in the denominator (<italic>N</italic> = 199). The I3 and PR6 values are determined by both productivity and citation rates. In this case, values for I3 and PR6 were virtually identical (<italic>r</italic> = 0.998; <italic>ρ</italic> = 0.966; <italic>p</italic> ≤ 0.01; <italic>N</italic> = 58).</p><p>Figure <xref rid="Fig4" ref-type="fig">4</xref> shows the relation between impact (I3) and size. Both <italic>Nano Letters</italic> and <italic>Nature Nanotechnology</italic> (indicated in red; grey in print) lie above the regression line, whereas the journal <italic>Mater Sci Eng A</italic>-<italic>Struct Mater</italic> does not. As shown in Table <xref rid="Tab1" ref-type="table">1</xref>, these deviations from the respective expectations are highly significant (<italic>p</italic> < 0.01). The curve also explicates that <italic>Nano Letters</italic> is a large journal, whereas <italic>Nature Nanotechnology</italic> is not very different in size and impact from a large number of specialist journals that are much smaller.<fig id="Fig4"><label>Fig. 4</label><caption><p>Regression of impact (I3) against the number of citable publications in 2007 and 2008</p></caption><graphic xlink:href="11192_2012_750_Fig4_HTML" id="MO6"></graphic></fig></p><p>Figure <xref rid="Fig5" ref-type="fig">5</xref> provides the results of the other statistical test, namely Dunn’s test for multiple comparisons. As noted, this test is limited in SPSS to 50 cases. I used the 50 journals with the highest IFs. Thirty-two of these journals belong to a core group (<italic>k</italic> = 29). This means that their citation patterns are not statistically different from at least (<italic>k</italic> − 1 =) 28 other journals in this group. Seven more journals are statistically homogenous with at least 25 other journals. However, the citation pattern of <italic>Nature Nanotechnology</italic> is completely different from all other journals in the group, while <italic>Nature Letters</italic> shares its citation pattern only with <italic>Nano Today</italic>, the journal with the second largest IF (=13.237) within this set, but again with a much lower I3 (of 0.18 %).<fig id="Fig5"><label>Fig. 5</label><caption><p>Homogenous groups in terms of distribution of citation distributions among 50 journals in “Nanoscience & nanotechnology.” An edge indicates similarity in the distribution at the 5 % level (<italic>p</italic> < 0.05/1225; using Bonferroni correction as explained in footnote 6)</p></caption><graphic xlink:href="11192_2012_750_Fig5_HTML" id="MO7"></graphic></fig></p><p>If we repeat this analysis with the distributions of I3—instead of citation distributions across these fifty journals (not shown), all journals are connected in a single graph (24 in a <italic>k</italic> = 17 core with 10 more journals connected to this component at the level of <italic>k</italic> = 11). However, the I3-distribution of <italic>Nature Nanotechnology</italic> is statistically similar only to the one of <italic>Nano Today</italic>, while this latter journal is connected to five other, among them <italic>Nano Letters.</italic> Not surprisingly, the impact pattern of <italic>Nano Letters</italic> is not otherwise different from <italic>ACS Nano</italic>, another journal of the American Chemical Society. <italic>ACS Nano</italic> was ranked 17th in Table <xref rid="Tab1" ref-type="table">1</xref>, and is another highly specialized journal with a relatively high IF (7.493), but a lower value of I3.</p><p>In summary, I3 and the IF measure very different aspects of impact. Whereas I3 can be considered as a measure of impact comparable to the <italic>h</italic>-index, but sophisticated in its statistics and elaboration, the IF is based on a division by the number of citable items in the denominator. Bensman (<xref ref-type="bibr" rid="CR2">2007</xref>) noted that Garfield thus wished to distinguish this indicator from “total cites” in order to control for the size of journals. However, the choice of the mean as a measure for impact has unfortunately led to the mistaken assumption that impact can be measured by averaging.</p></sec><sec id="Sec6"><title>Evaluation of nations</title><p>Because I3 is not an average over a distribution, but a summation of values of times cited normalized at the level of individual papers, aggregations other than those in terms of journals are equally possible. Since Narin (<xref ref-type="bibr" rid="CR40">1976</xref>) and Small and Garfield (<xref ref-type="bibr" rid="CR59">1985</xref>) one has been inclined to consider the two major aggregations in terms of journals and nations as the orthogonal axes of a matrix for citation analysis; but sets can also be aggregated using criteria such as keywords or author groups across journals and/or institutional addresses. Each paper has a percentile rank with reference to the set which represents the relevant field. Each subset can be tested on the significance of the contribution, that is, above or below statistical expectation.</p><p>Table <xref rid="Tab2" ref-type="table">2</xref> ranks the 20 leading nations (based on “integer counting” attributing a full point to each address).<xref ref-type="fn" rid="Fn8">8</xref> Table <xref rid="Tab3" ref-type="table">3</xref> disaggregates further to the level of institutions. Although the contributions of the Asian countries are evaluated in Table <xref rid="Tab2" ref-type="table">2</xref> as significantly below expectation (with the exception of Singapore), Table <xref rid="Tab3" ref-type="table">3</xref> informs us that individual institutions in these countries may perform significantly above expectation in terms of their impact. The Chinese Academy of Sciences in Beijing leads the list with over 60 % more impact than the second and third (American) universities. Except for the National University of Singapore, the contributions of non-Chinese centers in Asia rank as less significant or even (sometimes significantly) below expectation.<table-wrap id="Tab2"><label>Table 2</label><caption><p>Rankings among top-20 countries in terms of I3 (expressed as percentages of the sum) compared with number of publications, total citations, and % I3 (PR6) for the 58 journals in the ISI Subject Category “Nanoscience & nanotechnology”</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left"></th><th align="left">Publications</th><th align="left">Citations</th><th align="left">Cit/Pub</th><th align="left">% I3</th><th align="left">% PR6</th></tr></thead><tbody><tr><td align="left">1</td><td align="left">USA</td><td char="." align="char">16,039</td><td char="." align="char">234,370</td><td char="." align="char">14.61</td><td char="." align="char">27.73<sup>++</sup></td><td char="." align="char">27.93<sup>++</sup></td></tr><tr><td align="left">2</td><td align="left">Peoples R China</td><td char="." align="char">8,653</td><td char="." align="char">83,079</td><td char="." align="char">9.60</td><td char="." align="char">12.88<sup>−</sup></td><td char="." align="char">12.74<sup>−</sup></td></tr><tr><td align="left">3</td><td align="left">Japan</td><td char="." align="char">4,798</td><td char="." align="char">39,469</td><td char="." align="char">8.23</td><td char="." align="char">7.08<sup>−</sup></td><td char="." align="char">7.00<sup>−</sup></td></tr><tr><td align="left">4</td><td align="left">Germany</td><td char="." align="char">3,802</td><td char="." align="char">43,231</td><td char="." align="char">11.37</td><td char="." align="char">6.54<sup>++</sup></td><td char="." align="char">6.46<sup>++</sup></td></tr><tr><td align="left">5</td><td align="left">South Korea</td><td char="." align="char">4,250</td><td char="." align="char">32,772</td><td char="." align="char">7.71</td><td char="." align="char">6.10<sup>−</sup></td><td char="." align="char">6.14<sup>−</sup></td></tr><tr><td align="left">6</td><td align="left">France</td><td char="." align="char">2,803</td><td char="." align="char">21,099</td><td char="." align="char">7.53</td><td char="." align="char">4.04<sup>−</sup></td><td char="." align="char">3.92<sup>−</sup></td></tr><tr><td align="left">7</td><td align="left">England</td><td char="." align="char">1,963</td><td char="." align="char">24,875</td><td char="." align="char">12.67</td><td char="." align="char">3.24<sup>++</sup></td><td char="." align="char">3.24<sup>++</sup></td></tr><tr><td align="left">8</td><td align="left">Taiwan</td><td char="." align="char">2,529</td><td char="." align="char">19,068</td><td char="." align="char">7.54</td><td char="." align="char">3.24<sup>−</sup></td><td char="." align="char">3.30<sup>−</sup></td></tr><tr><td align="left">9</td><td align="left">Italy</td><td char="." align="char">2,021</td><td char="." align="char">15,738</td><td char="." align="char">7.79</td><td char="." align="char">3.00<sup>−</sup></td><td char="." align="char">2.92<sup>−</sup></td></tr><tr><td align="left">10</td><td align="left">India</td><td char="." align="char">2,215</td><td char="." align="char">13,593</td><td char="." align="char">6.14</td><td char="." align="char">2.67<sup>−</sup></td><td char="." align="char">2.72<sup>–</sup></td></tr><tr><td align="left">11</td><td align="left">Spain</td><td char="." align="char">1,527</td><td char="." align="char">14,692</td><td char="." align="char">9.62</td><td char="." align="char">2.45<sup>++</sup></td><td char="." align="char">2.33<sup>−</sup></td></tr><tr><td align="left">12</td><td align="left">Canada</td><td char="." align="char">1,360</td><td char="." align="char">13,418</td><td char="." align="char">9.87</td><td char="." align="char">2.09<sup>++</sup></td><td char="." align="char">2.04<sup>++</sup></td></tr><tr><td align="left">13</td><td align="left">Singapore</td><td char="." align="char">1,219</td><td char="." align="char">13,785</td><td char="." align="char">11.31</td><td char="." align="char">1.94<sup>++</sup></td><td char="." align="char">1.91<sup>++</sup></td></tr><tr><td align="left">14</td><td align="left">Netherlands</td><td char="." align="char">872</td><td char="." align="char">10,951</td><td char="." align="char">12.56</td><td char="." align="char">1.53<sup>++</sup></td><td char="." align="char">1.51<sup>++</sup></td></tr><tr><td align="left">15</td><td align="left">Australia</td><td char="." align="char">919</td><td char="." align="char">9,560</td><td char="." align="char">10.40</td><td char="." align="char">1.51<sup>++</sup></td><td char="." align="char">1.45<sup>++</sup></td></tr><tr><td align="left">16</td><td align="left">Switzerland</td><td char="." align="char">682</td><td char="." align="char">9,565</td><td char="." align="char">14.02</td><td char="." align="char">1.29<sup>++</sup></td><td char="." align="char">1.27<sup>++</sup></td></tr><tr><td align="left">17</td><td align="left">Belgium</td><td char="." align="char">668</td><td char="." align="char">5,683</td><td char="." align="char">8.51</td><td char="." align="char">1.05<sup>++</sup></td><td char="." align="char">1.04<sup>++</sup></td></tr><tr><td align="left">18</td><td align="left">Sweden</td><td char="." align="char">566</td><td char="." align="char">6,661</td><td char="." align="char">11.77</td><td char="." align="char">0.96<sup>++</sup></td><td char="." align="char">0.96<sup>++</sup></td></tr><tr><td align="left">19</td><td align="left">Israel</td><td char="." align="char">525</td><td char="." align="char">5,662</td><td char="." align="char">10.78</td><td char="." align="char">0.86<sup>++</sup></td><td char="." align="char">0.78<sup>++</sup></td></tr><tr><td align="left">20</td><td align="left">Russia</td><td char="." align="char">1,023</td><td char="." align="char">3,410</td><td char="." align="char">3.33</td><td char="." align="char">0.81<sup>− −</sup></td><td char="." align="char">1.11<sup>−</sup></td></tr></tbody></table><table-wrap-foot><p><sup>++</sup><italic>p</italic> < 0.01 above the expectation; <sup>+</sup> <italic>p</italic> < 0.05 above the expectation; <sup>− −</sup> <italic>p</italic> < 0.01 below the expectation; <sup>−</sup> <italic>p</italic> < 0.05 below the expectation</p></table-wrap-foot></table-wrap><table-wrap id="Tab3"><label>Table 3</label><caption><p>Rankings among the 20 top-institutes in terms of I3 (expressed as percentages of the sum) compared with number of publications and citations, and % I3 (PR6) for the 58 journals in the ISI Subject Category “Nanoscience & nanotechnology”</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left"></th><th align="left">Publications</th><th align="left">Citations</th><th align="left"><italic>c</italic>/<italic>p</italic></th><th align="left">% I3</th><th align="left">% PR6</th></tr></thead><tbody><tr><td align="left">1</td><td align="left">Chinese Acad Sci, Beijing, Peoples R China</td><td char="." align="char">824</td><td char="." align="char">11,258</td><td char="." align="char">13.66</td><td char="." align="char">1.43<sup>++</sup></td><td char="." align="char">1.42<sup>++</sup></td></tr><tr><td align="left">2</td><td align="left">MIT, Cambridge, USA</td><td char="." align="char">420</td><td char="." align="char">8,372</td><td char="." align="char">19.93</td><td char="." align="char">0.89<sup>++</sup></td><td char="." align="char">0.90<sup>++</sup></td></tr><tr><td align="left">3</td><td align="left">Univ Calif Berkeley, Berkeley, USA</td><td char="." align="char">425</td><td char="." align="char">8,639</td><td char="." align="char">20.33</td><td char="." align="char">0.88<sup>++</sup></td><td char="." align="char">0.92<sup>++</sup></td></tr><tr><td align="left">4</td><td align="left">Natl Univ Singapore, Singapore, Singapore</td><td char="." align="char">470</td><td char="." align="char">6,483</td><td char="." align="char">13.79</td><td char="." align="char">0.83<sup>++</sup></td><td char="." align="char">0.82<sup>++</sup></td></tr><tr><td align="left">5</td><td align="left">Northwestern Univ, Evanston, USA</td><td char="." align="char">304</td><td char="." align="char">7,594</td><td char="." align="char">24.98</td><td char="." align="char">0.63<sup>++</sup></td><td char="." align="char">0.68<sup>++</sup></td></tr><tr><td align="left">6</td><td align="left">Univ Michigan, Ann Arbor, USA</td><td char="." align="char">301</td><td char="." align="char">5,633</td><td char="." align="char">18.71</td><td char="." align="char">0.63<sup>++</sup></td><td char="." align="char">0.63<sup>++</sup></td></tr><tr><td align="left">7</td><td align="left">Univ Illinois, Urbana, USA</td><td char="." align="char">307</td><td char="." align="char">6,905</td><td char="." align="char">22.49</td><td char="." align="char">0.61<sup>++</sup></td><td char="." align="char">0.62<sup>++</sup></td></tr><tr><td align="left">8</td><td align="left">Georgia Inst Technol, Atlanta, USA</td><td char="." align="char">307</td><td char="." align="char">5,658</td><td char="." align="char">18.43</td><td char="." align="char">0.61<sup>++</sup></td><td char="." align="char">0.62<sup>++</sup></td></tr><tr><td align="left">9</td><td align="left">Nanyang Technol Univ, Singapore, Singapore</td><td char="." align="char">383</td><td char="." align="char">4,110</td><td char="." align="char">10.73</td><td char="." align="char">0.60<sup>++</sup></td><td char="." align="char">0.59<sup>++</sup></td></tr><tr><td align="left">10</td><td align="left">Penn State Univ, University Park, USA</td><td char="." align="char">293</td><td char="." align="char">4,519</td><td char="." align="char">15.42</td><td char="." align="char">0.52<sup>++</sup></td><td char="." align="char">0.53<sup>++</sup></td></tr><tr><td align="left">11</td><td align="left">Univ Cambridge, Cambridge, UK</td><td char="." align="char">276</td><td char="." align="char">4,677</td><td char="." align="char">16.95</td><td char="." align="char">0.52<sup>++</sup></td><td char="." align="char">0.52<sup>++</sup></td></tr><tr><td align="left">12</td><td align="left">Seoul Natl Univ, Seoul, South Korea</td><td char="." align="char">314</td><td char="." align="char">4,135</td><td char="." align="char">13.17</td><td char="." align="char">0.51<sup>++</sup></td><td char="." align="char">0.51<sup>++</sup></td></tr><tr><td align="left">13</td><td align="left">Peking Univ, Beijing, Peoples R China</td><td char="." align="char">285</td><td char="." align="char">3,698</td><td char="." align="char">12.98</td><td char="." align="char">0.50<sup>++</sup></td><td char="." align="char">0.50<sup>++</sup></td></tr><tr><td align="left">14</td><td align="left">Rice Univ, Houston, USA</td><td char="." align="char">239</td><td char="." align="char">6,170</td><td char="." align="char">25.82</td><td char="." align="char">0.49<sup>++</sup></td><td char="." align="char">0.55<sup>++</sup></td></tr><tr><td align="left">15</td><td align="left">Purdue Univ, Lafayette, USA</td><td char="." align="char">282</td><td char="." align="char">3,931</td><td char="." align="char">13.94</td><td char="." align="char">0.48<sup>++</sup></td><td char="." align="char">0.48<sup>++</sup></td></tr><tr><td align="left">16</td><td align="left">Tohoku Univ, Miyagi, Japan</td><td char="." align="char">349</td><td char="." align="char">2,274</td><td char="." align="char">6.52</td><td char="." align="char">0.47<sup>− −</sup></td><td char="." align="char">0.47<sup>− −</sup></td></tr><tr><td align="left">17</td><td align="left">Univ Washington, Seattle, USA</td><td char="." align="char">210</td><td char="." align="char">5,126</td><td char="." align="char">24.41</td><td char="." align="char">0.47<sup>++</sup></td><td char="." align="char">0.50<sup>++</sup></td></tr><tr><td align="left">18</td><td align="left">Natl Inst Mat Sci, Ibaraki, Japan</td><td char="." align="char">271</td><td char="." align="char">2,831</td><td char="." align="char">10.45</td><td char="." align="char">0.46<sup>++</sup></td><td char="." align="char">0.44<sup>++</sup></td></tr><tr><td align="left">19</td><td align="left">Natl Cheng Kung Univ, Tainan, Taiwan</td><td char="." align="char">340</td><td char="." align="char">2,256</td><td char="." align="char">6.64</td><td char="." align="char">0.43<sup>− −</sup></td><td char="." align="char">0.43<sup>− −</sup></td></tr><tr><td align="left">20</td><td align="left">Stanford Univ, Stanford, USA</td><td char="." align="char">200</td><td char="." align="char">5,894</td><td char="." align="char">29.47</td><td char="." align="char">0.42<sup>++</sup></td><td char="." align="char">0.47<sup>++</sup></td></tr></tbody></table><table-wrap-foot><p><sup>++</sup><italic>p</italic> < 0.01 above the expectation; <sup>+</sup> <italic>p</italic> < 0.05 above the expectation; <sup>− −</sup> <italic>p</italic> < 0.01 below the expectation; <sup>−</sup> <italic>p</italic> < 0.05 below the expectation</p></table-wrap-foot></table-wrap></p></sec><sec id="Sec7"><title>More specific delineation of the nano-set</title><p>Because of its interdisciplinarity, the WoS Subject Category “Nanoscience & nanotechnology” can be considered as a mixed bag (Leydesdorff and Rafols <xref ref-type="bibr" rid="CR34">2012</xref>; Rafols et al. <xref ref-type="bibr" rid="CR51">2010</xref>). Factor analysis of the citation matrix of these 58 journals in terms of the cited patterns provided us with a first factor showing robustly the same 15 journals with factor loadings >0.4 using both four or five factors for the extraction. This factor 1 can be considered as the representation of a more homogenous set of journals which publish about nanoscience and nanotechnology from the perspective of condensed matter physics and chemistry-based nanotechnology. Other factors are focused on micro- and nanofluidics (factor 2), nano-medicine (factor 3), and nano-electronics (factor 4; Ismael Rafols, personal communication, 20 March 2011). Factor 1 explains approx. 36 % of the variance in the matrix of aggregated citation relations among the 58 journals under study.</p><p>Let us take this restricted domain of 15 journals (<italic>N</italic> = 14,794 citable items)<xref ref-type="fn" rid="Fn9">9</xref> as a specific and typical research specialty (cf. Huang et al. <xref ref-type="bibr" rid="CR21">2011</xref>). Table <xref rid="Tab4" ref-type="table">4</xref> provides a summary of the contributions of journals, and the rankings of (top) countries, cities, and institutes in this domain. The percentiles were recalculated using these 15 journals as the reference set. In the first column of Table <xref rid="Tab4" ref-type="table">4</xref>, for example, the relative impact of <italic>Advances in Materials</italic> is greater than that of <italic>Nanotechnology</italic>, reversing their ranking (based on 58 journals) as shown in Table <xref rid="Tab1" ref-type="table">1</xref>. Thus, <italic>Nanotechnology</italic> is more highly cited outside this subset than <italic>Advances in Materials</italic>. The indicators of significance are, however, largely the same in both tables.<table-wrap id="Tab4"><label>Table 4</label><caption><p>Integrated impact (% I3) of journals and rankings of (15 top) countries, cities, and institutes in the domain of the 15 journals (in column a) more specifically selected as condensed-matter and chemistry-based nanotechnology</p></caption><table frame="hsides" rules="groups"><thead><tr><th align="left"></th><th align="left">Journals (<italic>N</italic> = 15)<break></break>(a)</th><th align="left">Countries (<italic>N</italic> = 95)<break></break>(b)</th><th align="left">Cities (<italic>N</italic> = 1,694)<break></break>(c)</th><th align="left">Institutes (<italic>N</italic> = 5,726)<break></break>(d)</th></tr></thead><tbody><tr><td align="left">1</td><td align="left"><italic>J Phys Chem C</italic><sup>− −</sup></td><td align="left">USA<sup>++</sup></td><td align="left">Beijing, Peoples R China<sup>++</sup></td><td align="left">Chinese Acad Sci, Beijing, Peoples R China<sup>++</sup></td></tr><tr><td align="left">2</td><td align="left"><italic>Nano Lett</italic><sup>++</sup></td><td align="left">Peoples R China<sup>− −</sup></td><td align="left">Singapore, Singapore</td><td align="left">Univ Calif Berkeley, Berkeley, USA<sup>++</sup></td></tr><tr><td align="left">3</td><td align="left"><italic>Advan Mater</italic><sup>++</sup></td><td align="left">Germany<sup>++</sup></td><td align="left">Seoul, South Korea<sup>− −</sup></td><td align="left">MIT, Cambridge, USA<sup>++</sup></td></tr><tr><td align="left">4</td><td align="left"><italic>Nanotechnology</italic><sup>− −</sup></td><td align="left">Japan<sup>− −</sup></td><td align="left">Cambridge, USA<sup>++</sup></td><td align="left">Northwestern Univ, Evanston, USA<sup>++</sup></td></tr><tr><td align="left">5</td><td align="left"><italic>Adv Funct Mater</italic><sup>++</sup></td><td align="left">South Korea</td><td align="left">Berkeley, USA<sup>++</sup></td><td align="left">Natl Univ Singapore, Singapore, Singapore<sup>++</sup></td></tr><tr><td align="left">6</td><td align="left"><italic>Physica E</italic><sup>− −</sup></td><td align="left">UK</td><td align="left">Shanghai, Peoples R China<sup>− −</sup></td><td align="left">Univ Michigan, Ann Arbor, USA<sup>++</sup></td></tr><tr><td align="left">7</td><td align="left"><italic>Small</italic><sup>++</sup></td><td align="left">France<sup>− −</sup></td><td align="left">Ibaraki, Japan</td><td align="left">Georgia Inst Technol, Atlanta, USA<sup>++</sup></td></tr><tr><td align="left">8</td><td align="left"><italic>ACS Nano</italic><sup>++</sup></td><td align="left">Italy<sup>− −</sup></td><td align="left">Tokyo, Japan<sup>− −</sup></td><td align="left">Univ Illinois, Urbana, USA<sup>++</sup></td></tr><tr><td align="left">9</td><td align="left"><italic>Nat Nanotechnol</italic><sup>++</sup></td><td align="left">Taiwan<sup>–</sup></td><td align="left">Evanston, USA<sup>++</sup></td><td align="left">Rice Univ, Houston, USA<sup>++</sup></td></tr><tr><td align="left">10</td><td align="left"><italic>Nano</italic><sup>− −</sup></td><td align="left">Spain<sup>− −</sup></td><td align="left">Atlanta, USA<sup>++</sup></td><td align="left">Peking Univ, Beijing, Peoples R China<sup>++</sup></td></tr><tr><td align="left">11</td><td align="left"><italic>Nano Res</italic><sup>−</sup></td><td align="left">Canada<sup>− −</sup></td><td align="left">Houston, USA<sup>++</sup></td><td align="left">Univ Cambridge, Cambridge, UK<sup>++</sup></td></tr><tr><td align="left">12</td><td align="left"><italic>Int J Nanotechnol</italic><sup>− −</sup></td><td align="left">Singapore</td><td align="left">Changchun, Peoples R China<sup>++</sup></td><td align="left">Univ Washington, Seattle, USA<sup>++</sup></td></tr><tr><td align="left">13</td><td align="left"><italic>Nano Today</italic><sup>++</sup></td><td align="left">India<sup>− −</sup></td><td align="left">Taipei, Taiwan</td><td align="left">Penn State Univ, University, USA<sup>++</sup></td></tr><tr><td align="left">14</td><td align="left"><italic>J Nanomater</italic><sup>− −</sup></td><td align="left">Australia<sup>−</sup></td><td align="left">Hong Kong, Peoples R China</td><td align="left">Nanyang Technol Univ, Singapore, Singapore<sup>−</sup></td></tr><tr><td align="left">15</td><td align="left"><italic>Plasmonics</italic><sup>− −</sup></td><td align="left">Netherlands<sup>++</sup></td><td align="left">Ann Arbor, USA<sup>++</sup></td><td align="left">Univ Calif Los Angeles, Los Angeles, USA<sup>++</sup></td></tr></tbody></table><table-wrap-foot><p><sup>++</sup><italic>p</italic> < 0.01 above the expectation; <sup>+</sup><italic>p</italic> < 0.05 above the expectation; <sup>– –</sup><italic>p</italic> < 0.01 below the expectation; <sup>–</sup><italic>p</italic> < 0.05 below the expectation</p></table-wrap-foot></table-wrap></p><p>The table confirms the impression obtained above: Asian units are more concentrated in large metropoles; at the higher level of aggregation of countries the USA is still dominant; and China performs below expectation because of lower citation rates. Except for the UK (University of Cambridge), European institutes and cities do not appear to play a leading role. Japanese cities and centers tend to score below expectation, but several Chinese centers are leading along with American ones.</p><p>One can use Dunn’s test (as above in Fig. <xref rid="Fig4" ref-type="fig">4</xref>) for the evaluation, for example, of the extent to which the impact-profiles of the 15 institutions are significantly different and/or homogenous. The I3-distributions of ten American universities and the University of Cambridge in the UK are statistically homogenous. Three of these universities (Rice University, Penn State, the University of Cambridge, UK) and the National University of Singapore have stronger roles at the interface with the otherwise differently profiled Chinese universities. In other words, the rank-ordering among the American-British universities in this core set is also an effect of size, as it is among the Asian ones. If we use citations instead of I3 values for Dunn’s test, the results are not different except that the University of California in Los Angeles (number 15 in rank) is no longer included among the core set of American/UK universities. Thus, the analysis in terms of I3 refines our ability to compare citation distributions involving multiple comparisons.</p><p>What happened to Western Europe? With the exception of Cambridge (UK), it is absent from the listing of institutions in Table <xref rid="Tab4" ref-type="table">4</xref>, but Germany, for example, takes the second position in the ranking of nations. Figure <xref rid="Fig6" ref-type="fig">6</xref> cuts the northwestern section of Europe out of a global map (available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/nano2011/nano2011.htm">http://www.leydesdorff.net/nano2011/nano2011.htm</ext-link>) using this same data and using methods developed previously (Bornmann and Leydesdorff <xref ref-type="bibr" rid="CR5">2011</xref>; Leydesdorff and Persson <xref ref-type="bibr" rid="CR70">2010</xref>). Bornmann and Leydesdorff (<xref ref-type="bibr" rid="CR5">2011</xref>) used the top-10 % as an indicator of excellence, but one can use I3 values without setting a threshold for indicating whether impact is above or below expectation.<fig id="Fig6"><label>Fig. 6</label><caption><p>Distribution of cities in North-Western Europe above or below expectations in terms of integrated impact on the basis of 15 core journals. The sizes of the nodes are proportional to the logarithm of the number of papers; colors: <italic>red</italic> indicates below expectation; <italic>green</italic> above. The Google Map is available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/nano2011/nano2011.htm">http://www.leydesdorff.net/nano2011/nano2011.htm</ext-link>. (Color figure online)</p></caption><graphic xlink:href="11192_2012_750_Fig6_HTML" id="MO8"></graphic></fig></p><p>Figure <xref rid="Fig6" ref-type="fig">6</xref> shows that many cities other than Cambridge, UK, perform significantly above expectation, in Germany, for example. These, however, are often smaller towns with smaller universities, but at the aggregated level of countries Germany is ranked third and significantly above expectation, whereas the UK is ranked sixth and not different from expectation. The larger map informs us that papers from France and Mediterranean countries on average have less impact in this field than expected.</p></sec></sec><sec id="Sec8"><title>Specification of expectation</title><p>Because I3—and <italic>mutatis mutandis</italic> PR6—is based on a summation, the values for units of analysis such as countries or journals can be large. For example, the I3-value of <italic>Nano Letters</italic> in the set of 58 journals was 117,450, representing only 8 % of the total I3 of the set, namely 1,469,253. For the expected value of I3, this latter value was normalized using the share of publications in the set (<italic>n</italic><sub><italic>i</italic></sub>/∑<sub><italic>i</italic></sub><italic>n</italic><sub><italic>i</italic></sub> = 1,507/31,644). Therefore, in this case: exp(I3) = 1,469,253 * (1,507/31,644) = 69,971. The differences between observed and expected values can thus be large; this so easily generates significance using the <italic>z</italic>-test that the test perhaps loses its discriminating value. The measure I3 is very sensitive to differences in performance.<xref ref-type="fn" rid="Fn10">10</xref></p><p>An obvious alternative would be to test the percentage I3 for each unit of analysis against the percentage of publications.<xref ref-type="fn" rid="Fn11">11</xref> However, I3 is dependent on the <italic>N</italic> of publications, and percentages (or permillages) may be small in the case of cities or institutes (<0.1 %). Using relative publication rates as expected values, for example, only <italic>Nano Letters</italic> and <italic>Advances in Materials</italic> were flagged as impacting significantly above expectation in Table <xref rid="Tab1" ref-type="table">1</xref> (<italic>p</italic> ≤ 0.01), but using this alternative option none of the smaller journals remain significantly above or below expectations. In Table <xref rid="Tab4" ref-type="table">4</xref>, all significance indications for countries, cities, and institutes would disappear, and the Google Map accordingly would become uniformative. This alternative is therefore unattractive.</p><p>A third option could be to compare the observed impact <italic>per paper</italic> with the expected one per paper. This specification prevents the accumulation of I3 values and thus differences from expected values. The expected value of I3/paper on the basis of the set could then also be considered as a “world average.” Instead of dividing the observed average by the expected one [as in the case of MOCR/MECR (Budapest), NMCR (Louvain), or the “old Leiden crown indicator”], the expected value could be used as a yardstick for the testing. However, one would expect the results to suffer from the same problems as the previous use of CPPs, namely, that the <italic>N</italic> of papers is used in the denominator so that larger units of analysis are relatively disadvantaged and smaller units foregrounded.</p><p>One can draw the same map as Fig. <xref rid="Fig6" ref-type="fig">6</xref>, but using this indicator (not shown here, but available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/nano2011/nano2011b.htm">http://www.leydesdorff.net/nano2011/nano2011b.htm</ext-link>). Indeed, Cambridge UK (which is sized as before; <italic>N</italic> = 228) is no longer flagged as impacting significantly above expectation. Smaller centers such as Ludwigshaven in Germany (<italic>N</italic> = 9) and Maastricht in the Netherlands (<italic>N</italic> = 8) are now flagged as performing significantly above expectation. (A minimum of <italic>N</italic> = 5 was used for the test for reasons of reliability.)</p><p>Nevertheless, this map provides another view to the same data, and I decided to include it in the routine at the Internet. In addition to ztest.txt, a file ri3r.txt is generated by i3cit2.exe (and i3inst2.exe, respectively) which enables the user to draw this map. The z-score is also saved in ucities.dbf with the fieldname “zaverage.”<xref ref-type="fn" rid="Fn12">12</xref> Unlike previous indicators (derived from RCR = MOCR/MECR; Schubert and Braun <xref ref-type="bibr" rid="CR54">1986</xref>), this indicator is not based on a division of means, but tests citations in terms of the percentile scores of the cited publications against a “world average” as the expectation. Let me call this indicator the <italic>Relative I3</italic>-<italic>Rate</italic> (RI3R) in honour of the RCR which has served as an indicator for 25 years.</p><p>In summary, one can distinguish between testing integrated (I3) and average (I3/<italic>n</italic>) citation impact. Integrated impact is more easily significantly different from expectation than average impact. Furthermore, the latter is also determined by the <italic>n</italic> in the denominator.</p><p>Let me finally note that one could further inform the expectation depending on the research question (Cassidy Sugimoto, personal communication, August 16, 2011). For example, one could assume that the chances of being cited are not equal for established authors and newcomers, and thus weigh publications according to the specification of this expectation. The statistical tests remain the same, but the values of the expectations are then different.</p></sec><sec id="Sec9"><title>Conclusions and discussion</title><p>The integrated impact indicator I3 provides us with a versatile instrument which (i) can be applied to institutional units of analysis and journals (or other document sets); (ii) takes both publication and citation rates into account like the <italic>h</italic>-index; and (iii) enables us to use non-parametric statistics on skewed distributions as against using averages. The values of I3 in terms of percentiles can (iv) be summed in evaluative schemes such as the six ranks used by the NSF (and otherwise; e.g. quartiles).</p><p>The first requirement is a careful decision about the reference set. This set can be based on a sophisticated search string in the Web of Science or on a delineated journal set. One can vary the citation windows and use fractional instead of integer counting. The measure is formalized much more abstractly, namely by reducing any set to one hundred percentiles and thus making unequal distributions comparable without giving up the notion of quality either by abandoning the tales of the distribution (as in the case of the <italic>h</italic>-index) or by using “total cites” instead of more sophisticated statistics.</p><p>Let me emphasize that the new measure does not prevent one from taking an average citation over publication rate as another measure. However, some of the issues in the debate over impact indicators that has raged for the past 2 years can be solved by using non-parametric statistics on which this new measure is based. I would particularly recommend its use combined with fractional citation counting as proposed by Leydesdorff and Opthof (<xref ref-type="bibr" rid="CR33">2010</xref>) and hitherto applied in a limited number of studies (Leydesdorff and Bornmann <xref ref-type="bibr" rid="CR28">2011a</xref>; Leydesdorff and Shin <xref ref-type="bibr" rid="CR35">2011</xref>; Prathap <xref ref-type="bibr" rid="CR45">2011</xref>; Zhou and Leydesdorff <xref ref-type="bibr" rid="CR68">2011</xref>; Rafols et al. in press; cf. Radicchi and Castellano <xref ref-type="bibr" rid="CR48">2012</xref>). Fractional counting would correct for disciplinary structures (and biases) in the potentially interdisciplinary reference sets. In the case of nanoscience and nanotechnology, I have shown how one can identify 15 journals (out of 58) that are specific for condensed-matter and chemistry-based nanotechnology.</p><p>These analytical advantages can be combined with practicalities such as the straightforward option to show the results of an evaluation at the city or institute level by using Google Maps. As noted, fractional counting (in terms of authors and addresses) can be expected to further refine these maps. The software is available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/software/i3">http://www.leydesdorff.net/software/i3</ext-link>. This software routinely provides the I3 based on 100 percentiles as well as the six percentile classes distinguished above.</p></sec><sec id="Sec10"><title>Normative implications</title><p>The use of citation and publication analysis for evaluative purposes is a normative choice. Many good reasons can be given why one should not rely too much on statistics when taking decisions, particularly when the set is small, as in the case of individual hiring or promotion decisions. Peer review, however, also has constraints (e.g., Bornmann et al. <xref ref-type="bibr" rid="CR6">2010</xref>), and in the case of large sets the reading of actual papers may be too time-consuming. Bibliometric indicators can then serve as proxies provided that error can be specified and the illusion of clarity conveyed by quantification can be avoided.</p><p>In Leydesdorff et al. (<xref ref-type="bibr" rid="CR32">2011b</xref>, at pp. 1371f.), we specified a set of criteria for citation indicators, such as:<list list-type="bullet"><list-item><p>Citation-based indicators should accommodate various normative schemes such as the six categories of the NSF;</p></list-item><list-item><p>Citation-based indicators should be applicable to different sets of reference such as fields of science, nations, institutions, etc.;</p></list-item><list-item><p>The indicator should allow productivity to be taken into account. One should, for example, be able to compare two papers in the 39th percentile with a single one in the 78th percentile (perhaps after weighing in an evaluation scheme);</p></list-item><list-item><p>The indicator should result in a rather straightforward measure such as a percentage of maximum performance;</p></list-item><list-item><p>Error estimation and statistics should enable the analyst to specify the significance of individual contributions and differences in rankings.</p></list-item></list></p><p>By developing the apparatus of I3 and showing its comparability with the NSF scheme of six ranks, its decomposability in terms of percentages of contributions, and its relative straightforwardness in being based on percentiles in a distribution that allows for the use of non-parametric statistics, Leydesdorff and Bornmann (<xref ref-type="bibr" rid="CR29">2011b</xref>) have shown how one can meet these criteria. This study has elaborated the technique for a highly policy-relevant field of science, notably, “Nanoscience & nanotechnology,” and integrated I3 with the geographic evaluation of excellence using Google Maps (Bornmann and Leydesdorff <xref ref-type="bibr" rid="CR5">2011</xref>). The instrument is ready to be used.</p></sec></body><back><ack><p>I thank Lutz Bornmann for continuous discussion of these issues. I also wish to thank Cassidy Sugimoto, Ismael Rafols, and an anonymous referee for comments on a previous version of this paper. I am grateful to Thomson-Reuters for access to the data.</p><sec id="d29e3538"><title>Open Access</title><p>This article is distributed under the terms of the Creative Commons Attribution License which permits any use, distribution, and reproduction in any medium, provided the original author(s) and the source are credited.</p></sec></ack><ref-list id="Bib1"><title>References</title><ref id="CR2"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bensman</surname><given-names>SJ</given-names></name></person-group><article-title>Garfield and the impact factor</article-title><source>Annual Review of Information Science and Technology</source><year>2007</year><volume>41</volume><issue>1</issue><fpage>93</fpage><lpage>155</lpage><pub-id pub-id-type="doi">10.1002/aris.2007.1440410110</pub-id></mixed-citation></ref><ref id="CR3"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bensman</surname><given-names>SJ</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title>Definition and identification of journals as bibliographic and subject entities: librarianship vs. ISI Journal Citation Reports (JCR) methods and their effect on citation measures</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2009</year><volume>60</volume><issue>6</issue><fpage>1097</fpage><lpage>1117</lpage><pub-id pub-id-type="doi">10.1002/asi.21020</pub-id></mixed-citation></ref><ref id="CR4"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bonaccorsi</surname><given-names>A</given-names></name><name><surname>Vargas</surname><given-names>J</given-names></name></person-group><article-title>Proliferation dynamics in new sciences</article-title><source>Research Policy</source><year>2010</year><volume>39</volume><issue>8</issue><fpage>1034</fpage><lpage>1050</lpage><pub-id pub-id-type="doi">10.1016/j.respol.2010.05.002</pub-id></mixed-citation></ref><ref id="CR5"><mixed-citation publication-type="other">Bornmann, L., & Leydesdorff, L. (2011). Which cities produce excellent papers worldwide more than can be expected? A new mapping approach—using Google Maps—based on statistical significance testing. <italic>Journal of the American Society for Information Science and Technology</italic> (in press). <ext-link ext-link-type="uri" xlink:href="http://arxiv.org/abs/1103.3216">http://arxiv.org/abs/1103.3216</ext-link>.</mixed-citation></ref><ref id="CR6"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bornmann</surname><given-names>L</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Besselaar</surname><given-names>P</given-names></name></person-group><article-title>A meta-evaluation of scientific research proposals: Different ways of comparing rejected to awarded applications</article-title><source>Journal of Informetrics</source><year>2010</year><volume>4</volume><issue>3</issue><fpage>211</fpage><lpage>220</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2009.10.004</pub-id></mixed-citation></ref><ref id="CR7"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bornmann</surname><given-names>L</given-names></name><name><surname>Mutz</surname><given-names>R</given-names></name></person-group><article-title>Further steps towards an ideal method of measuring citation performance: The avoidance of citation (ratio) averages in field-normalization</article-title><source>Journal of Informetrics</source><year>2011</year><volume>5</volume><issue>1</issue><fpage>228</fpage><lpage>230</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2010.10.009</pub-id></mixed-citation></ref><ref id="CR8"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bornmann</surname><given-names>L</given-names></name><name><surname>Mutz</surname><given-names>R</given-names></name><name><surname>Neuhaus</surname><given-names>C</given-names></name><name><surname>Daniel</surname><given-names>HD</given-names></name></person-group><article-title>Citation counts for research evaluation: standards of good practice for analyzing bibliometric data and presenting and interpreting results</article-title><source>Ethics in Science and Environmental Politics (ESEP)</source><year>2008</year><volume>8</volume><issue>1</issue><fpage>93</fpage><lpage>102</lpage><pub-id pub-id-type="doi">10.3354/esep00084</pub-id></mixed-citation></ref><ref id="CR9"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Bornmann</surname><given-names>L</given-names></name><name><surname>Schier</surname><given-names>H</given-names></name><name><surname>Marx</surname><given-names>W</given-names></name><name><surname>Daniel</surname><given-names>HD</given-names></name></person-group><article-title>Is interactive open access publishing able to identify high impact submissions? A study on the predictive validity of Atmospheric Chemistry and Physics by using percentile rank classes</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>52</volume><issue>1</issue><fpage>61</fpage><lpage>71</lpage><pub-id pub-id-type="doi">10.1002/asi.21418</pub-id></mixed-citation></ref><ref id="CR10"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Boyack</surname><given-names>KW</given-names></name><name><surname>Klavans</surname><given-names>R</given-names></name></person-group><person-group person-group-type="editor"><name><surname>Noyons</surname><given-names>E</given-names></name><name><surname>Ngulube</surname><given-names>P</given-names></name><name><surname>Leta</surname><given-names>J</given-names></name></person-group><article-title>Multiple dimensions of journal specificity: Why journals can’t be assigned to disciplines</article-title><source>The 13th conference of the international society for scientometrics and informetrics</source><year>2011</year><publisher-loc>Durban, South Africa</publisher-loc><publisher-name>ISSI, Leiden University and the University of Zululand</publisher-name><fpage>123</fpage><lpage>133</lpage></mixed-citation></ref><ref id="CR11"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Chan</surname><given-names>LM</given-names></name></person-group><source>A guide to the library of congress classification</source><year>1999</year><edition>5</edition><publisher-loc>Englewood, CO</publisher-loc><publisher-name>Libraries Unlimited</publisher-name></mixed-citation></ref><ref id="CR12"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Egghe</surname><given-names>L</given-names></name></person-group><article-title>Averages of ratios compared to ratios of averages: Mathematical results</article-title><source>Journal of Informetrics</source><year>2012</year><volume>6</volume><issue>2</issue><fpage>307</fpage><lpage>317</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2011.12.007</pub-id></mixed-citation></ref><ref id="CR13"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><article-title>Citation analysis as a tool in journal evaluation</article-title><source>Science</source><year>1972</year><volume>178</volume><issue>4060</issue><fpage>471</fpage><lpage>479</lpage><pub-id pub-id-type="doi">10.1126/science.178.4060.471</pub-id><pub-id pub-id-type="pmid">5079701</pub-id></mixed-citation></ref><ref id="CR14"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><article-title>Is citation analysis a legitimate evaluation tool?</article-title><source>Scientometrics</source><year>1979</year><volume>1</volume><issue>4</issue><fpage>359</fpage><lpage>375</lpage><pub-id pub-id-type="doi">10.1007/BF02019306</pub-id></mixed-citation></ref><ref id="CR15"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><source>Citation indexing: Its theory and application in science, technology, and humanities</source><year>1979</year><publisher-loc>New York</publisher-loc><publisher-name>Wiley</publisher-name></mixed-citation></ref><ref id="CR16"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Gingras</surname><given-names>Y</given-names></name><name><surname>Larivière</surname><given-names>V</given-names></name></person-group><article-title>There are neither “king” nor “crown” in scientometrics: Comments on a supposed “alternative” method of normalization</article-title><source>Journal of Informetrics</source><year>2011</year><volume>5</volume><issue>1</issue><fpage>226</fpage><lpage>227</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2010.10.005</pub-id></mixed-citation></ref><ref id="CR17"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Glänzel</surname><given-names>W</given-names></name></person-group><article-title>Characteristic scores and scales: A bibliometric analysis of subject characteristics based on long-term citation observation</article-title><source>Journal of Informetrics</source><year>2007</year><volume>1</volume><issue>1</issue><fpage>92</fpage><lpage>102</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2006.10.001</pub-id></mixed-citation></ref><ref id="CR18"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Glänzel</surname><given-names>W</given-names></name><name><surname>Schubert</surname><given-names>A</given-names></name></person-group><article-title>A new classification scheme of science fields and subfields designed for scientometric evaluation purposes</article-title><source>Scientometrics</source><year>2003</year><volume>56</volume><issue>3</issue><fpage>357</fpage><lpage>367</lpage><pub-id pub-id-type="doi">10.1023/A:1022378804087</pub-id></mixed-citation></ref><ref id="CR19"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Glänzel</surname><given-names>W</given-names></name><name><surname>Thijs</surname><given-names>B</given-names></name><name><surname>Schubert</surname><given-names>A</given-names></name><name><surname>Debackere</surname><given-names>K</given-names></name></person-group><article-title>Subfield-specific normalized relative indicators and a new generation of relational charts: Methodological foundations illustrated on the assessment of institutional research performance</article-title><source>Scientometrics</source><year>2009</year><volume>78</volume><issue>1</issue><fpage>165</fpage><lpage>188</lpage><pub-id pub-id-type="doi">10.1007/s11192-008-2109-5</pub-id></mixed-citation></ref><ref id="CR20"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hirsch</surname><given-names>JE</given-names></name></person-group><article-title>An index to quantify an individual’s scientific research output</article-title><source>Proceedings of the National Academy of Sciences of the USA</source><year>2005</year><volume>102</volume><issue>46</issue><fpage>16569</fpage><lpage>16572</lpage><pub-id pub-id-type="doi">10.1073/pnas.0507655102</pub-id><pub-id pub-id-type="pmid">16275915</pub-id></mixed-citation></ref><ref id="CR71"><mixed-citation publication-type="other">Holm, S. (1979). A simple sequentially rejective multiple test procedure. <italic>Scandinavian Journal of Statistics</italic>, <italic>6</italic>(2), 65–70.</mixed-citation></ref><ref id="CR21"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Huang</surname><given-names>C</given-names></name><name><surname>Notten</surname><given-names>A</given-names></name><name><surname>Rasters</surname><given-names>N</given-names></name></person-group><article-title>Nanoscience and technology publications and patents: a review of social science studies and search strategies</article-title><source>The Journal of Technology Transfer</source><year>2011</year><volume>36</volume><issue>2</issue><fpage>145</fpage><lpage>172</lpage><pub-id pub-id-type="doi">10.1007/s10961-009-9149-8</pub-id></mixed-citation></ref><ref id="CR22"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Kamada</surname><given-names>T</given-names></name><name><surname>Kawai</surname><given-names>S</given-names></name></person-group><article-title>An algorithm for drawing general undirected graphs</article-title><source>Information Processing Letters</source><year>1989</year><volume>31</volume><issue>1</issue><fpage>7</fpage><lpage>15</lpage>9936290679.68128<pub-id pub-id-type="doi">10.1016/0020-0190(89)90102-6</pub-id></mixed-citation></ref><ref id="CR23"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Levine</surname><given-names>G</given-names></name></person-group><source>A guide to SPSS for analysis of variance</source><year>1991</year><publisher-loc>Hillsdale, NJ</publisher-loc><publisher-name>Lawrence Erlbaum</publisher-name></mixed-citation></ref><ref id="CR25"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title>Can scientific journals be classified in terms of aggregated journal-journal citation relations using the journal citation reports?</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2006</year><volume>57</volume><issue>5</issue><fpage>601</fpage><lpage>613</lpage><pub-id pub-id-type="doi">10.1002/asi.20322</pub-id></mixed-citation></ref><ref id="CR26"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title><italic>Caveats</italic> for the use of citation indicators in research and journal evaluation</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2008</year><volume>59</volume><issue>2</issue><fpage>278</fpage><lpage>287</lpage><pub-id pub-id-type="doi">10.1002/asi.20743</pub-id></mixed-citation></ref><ref id="CR27"><mixed-citation publication-type="other">Leydesdorff, L. (2012). Alternatives to the journal impact factor: I3 and the top-10% (or top-25%?) of the most-highly cited papers. <italic>Scientometrics.</italic> doi:10.1007/s11192-11012-10660-11196.</mixed-citation></ref><ref id="CR28"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Bornmann</surname><given-names>L</given-names></name></person-group><article-title>How fractional counting affects the impact factor: Normalization in terms of differences in citation potentials among fields of science</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>62</volume><issue>2</issue><fpage>217</fpage><lpage>229</lpage><pub-id pub-id-type="doi">10.1002/asi.21450</pub-id></mixed-citation></ref><ref id="CR29"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Bornmann</surname><given-names>L</given-names></name></person-group><article-title>Percentile ranks and the integrated impact indicator (<italic>I3</italic>)</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>62</volume><issue>11</issue><fpage>2133</fpage><lpage>2146</lpage><pub-id pub-id-type="doi">10.1002/asi.21609</pub-id></mixed-citation></ref><ref id="CR30"><mixed-citation publication-type="other">Leydesdorff, L., & Bornmann, L. (in press). Percentile ranks and the integrated impact indicator (<italic>I3</italic>). <italic>Journal of the American Society for Information Science and Technology</italic>. <ext-link ext-link-type="uri" xlink:href="http://arxiv.org/abs/1112.6281">http://arxiv.org/abs/1112.6281</ext-link>.</mixed-citation></ref><ref id="CR31"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Bornmann</surname><given-names>L</given-names></name><name><surname>Mutz</surname><given-names>R</given-names></name><name><surname>Opthof</surname><given-names>T</given-names></name></person-group><article-title>Turning the tables in citation analysis one more time: Principles for comparing sets of documents</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>62</volume><issue>7</issue><fpage>1370</fpage><lpage>1381</lpage><pub-id pub-id-type="doi">10.1002/asi.21534</pub-id></mixed-citation></ref><ref id="CR32"><mixed-citation publication-type="other">Leydesdorff, L., Hammarfelt, B., & Salah, A. A. A. (2011b). The structure of the arts & humanities citation index: A mapping on the basis of aggregated citations among 1,157 journals. <italic>Journal of the American Society for Information Science and Technology</italic>, <italic>62</italic>(12), 2414–2426.</mixed-citation></ref><ref id="CR33"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Opthof</surname><given-names>T</given-names></name></person-group><article-title><italic>Scopus</italic>’ source normalized impact per paper (SNIP) <italic>versus</italic> the journal impact factor based on fractional counting of citations</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2010</year><volume>61</volume><issue>11</issue><fpage>2365</fpage><lpage>2396</lpage><pub-id pub-id-type="doi">10.1002/asi.21371</pub-id></mixed-citation></ref><ref id="CR70"><mixed-citation publication-type="other">Leydesdorff, L., & Persson, O. (2010). Mapping the geography of science: Distribution patterns and networks of relations among cities and institutes. <italic>Journal of the American Society of Information Science and Technology,</italic><italic>61</italic>(8), 1622–1634.</mixed-citation></ref><ref id="CR34"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Rafols</surname><given-names>I</given-names></name></person-group><article-title>Interactive overlays: A new method for generating global journal maps from web-of-science data</article-title><source>Journal of Informetrics</source><year>2012</year><volume>6</volume><issue>3</issue><fpage>318</fpage><lpage>332</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2011.11.003</pub-id></mixed-citation></ref><ref id="CR35"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Leydesdorff</surname><given-names>L</given-names></name><name><surname>Shin</surname><given-names>JC</given-names></name></person-group><article-title>How to evaluate universities in terms of their relative citation impacts: Fractional counting of citations and the normalization of differences among disciplines</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>62</volume><issue>6</issue><fpage>1146</fpage><lpage>1155</lpage><pub-id pub-id-type="doi">10.1002/asi.21511</pub-id></mixed-citation></ref><ref id="CR36"><mixed-citation publication-type="other">Leydesdorff, L., Zhou, P., & Bornmann, L. (in preparation). How can impact factors be normalized across fields of science? An assessment of science citation index 2010 in terms of percentile ranks and fractional counts.</mixed-citation></ref><ref id="CR37"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Lundberg</surname><given-names>J</given-names></name></person-group><article-title>Lifting the crown—citation z-score</article-title><source>Journal of Informetrics</source><year>2007</year><volume>1</volume><issue>2</issue><fpage>145</fpage><lpage>154</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2006.09.007</pub-id></mixed-citation></ref><ref id="CR38"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Moed</surname><given-names>HF</given-names></name></person-group><article-title>Measuring contextual citation impact of scientific journals</article-title><source>Journal of Informetrics</source><year>2010</year><volume>4</volume><issue>3</issue><fpage>265</fpage><lpage>277</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2010.01.002</pub-id></mixed-citation></ref><ref id="CR39"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Moed</surname><given-names>HF</given-names></name><name><surname>Bruin</surname><given-names>RE</given-names></name><name><surname>Leeuwen</surname><given-names>TN</given-names></name></person-group><article-title>New bibliometric tools for the assessment of national research performance: Database description, overview of indicators and first applications</article-title><source>Scientometrics</source><year>1995</year><volume>33</volume><issue>3</issue><fpage>381</fpage><lpage>422</lpage><pub-id pub-id-type="doi">10.1007/BF02017338</pub-id></mixed-citation></ref><ref id="CR40"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Narin</surname><given-names>F</given-names></name></person-group><source>Evaluative bibliometrics: The use of publication and citation analysis in the evaluation of scientific activity</source><year>1976</year><publisher-loc>Washington, DC</publisher-loc><publisher-name>National Science Foundation</publisher-name></mixed-citation></ref><ref id="CR41"><mixed-citation publication-type="other">National Science Board. (2010). <italic>Science and engineering indicators</italic>. Washington DC: National Science Foundation. <ext-link ext-link-type="uri" xlink:href="http://www.nsf.gov/statistics/seind10/">http://www.nsf.gov/statistics/seind10/</ext-link>.</mixed-citation></ref><ref id="CR42"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nederhof</surname><given-names>AJ</given-names></name></person-group><article-title>Bibliometric monitoring of research performance in the social sciences and the humanities: A review</article-title><source>Scientometrics</source><year>2006</year><volume>66</volume><issue>1</issue><fpage>81</fpage><lpage>100</lpage>1710343<pub-id pub-id-type="doi">10.1007/s11192-006-0007-2</pub-id></mixed-citation></ref><ref id="CR43"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Opthof</surname><given-names>T</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title><italic>Caveats</italic> for the journal and field normalizations in the CWTS (“Leiden”) evaluations of research performance</article-title><source>Journal of Informetrics</source><year>2010</year><volume>4</volume><issue>3</issue><fpage>423</fpage><lpage>430</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2010.02.003</pub-id></mixed-citation></ref><ref id="CR45"><mixed-citation publication-type="other">Prathap, G. (2011). Fractionalized exergy for evaluating research performance. <italic>Journal of the American Society for Information Science and Technology,</italic><italic>62</italic>(11), 2294–2295.</mixed-citation></ref><ref id="CR46"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Pudovkin</surname><given-names>AI</given-names></name><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><article-title>Algorithmic procedure for finding semantically related journals</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2002</year><volume>53</volume><issue>13</issue><fpage>1113</fpage><lpage>1119</lpage><pub-id pub-id-type="doi">10.1002/asi.10153</pub-id></mixed-citation></ref><ref id="CR47"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Pudovkin</surname><given-names>AI</given-names></name><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><article-title>Percentile rank and author superiority indexes for evaluating individual journal articles and the author’s overall citation performance</article-title><source>CollNet Journal of Scientometrics and Information Management</source><year>2009</year><volume>3</volume><issue>2</issue><fpage>3</fpage><lpage>10</lpage></mixed-citation></ref><ref id="CR48"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Radicchi</surname><given-names>F</given-names></name><name><surname>Castellano</surname><given-names>C</given-names></name></person-group><article-title>Testing the fairness of citation indicators for comparison across scientific domains: the case of fractional citation counts</article-title><source>Journal of Informetrics</source><year>2012</year><volume>6</volume><issue>1</issue><fpage>121</fpage><lpage>130</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2011.09.002</pub-id></mixed-citation></ref><ref id="CR49"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rafols</surname><given-names>I</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title>Content-based and algorithmic classifications of journals: Perspectives on the dynamics of scientific communication and indexer effects</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2009</year><volume>60</volume><issue>9</issue><fpage>1823</fpage><lpage>1835</lpage><pub-id pub-id-type="doi">10.1002/asi.21086</pub-id></mixed-citation></ref><ref id="CR50"><mixed-citation publication-type="other">Rafols, I., Leydesdorff, L., O’Hare, A., Nightingale, P., & Stirling, A. (in press). How journal rankings can suppress interdisciplinary research: A comparison between innovation studies and business & management. <italic>Research Policy</italic>.</mixed-citation></ref><ref id="CR51"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rafols</surname><given-names>I</given-names></name><name><surname>Porter</surname><given-names>A</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title>Science overlay maps: A new tool for research policy and library management</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2010</year><volume>61</volume><issue>9</issue><fpage>1871</fpage><lpage>1887</lpage><pub-id pub-id-type="doi">10.1002/asi.21368</pub-id></mixed-citation></ref><ref id="CR52"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rousseau</surname><given-names>R</given-names></name></person-group><article-title>Basic properties of both percentile rank scores and the I3 indicator</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2012</year><volume>63</volume><issue>2</issue><fpage>416</fpage><lpage>420</lpage><pub-id pub-id-type="doi">10.1002/asi.21684</pub-id></mixed-citation></ref><ref id="CR53"><mixed-citation publication-type="other">Schreiber, M. (in press). Inconsistencies of Recently Proposed Citation Impact Indicators and how to Avoid Them. <italic>Journal of the American Society for Information Science and Technology</italic>. Available at arXiv:1202.3861.</mixed-citation></ref><ref id="CR54"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Schubert</surname><given-names>A</given-names></name><name><surname>Braun</surname><given-names>T</given-names></name></person-group><article-title>Relative indicators and relational charts for comparative assessment of publication output and citation impact</article-title><source>Scientometrics</source><year>1986</year><volume>9</volume><issue>5</issue><fpage>281</fpage><lpage>291</lpage><pub-id pub-id-type="doi">10.1007/BF02017249</pub-id></mixed-citation></ref><ref id="CR55"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Seglen</surname><given-names>PO</given-names></name></person-group><article-title>The skewness of science</article-title><source>Journal of the American Society for Information Science</source><year>1992</year><volume>43</volume><issue>9</issue><fpage>628</fpage><lpage>638</lpage><pub-id pub-id-type="doi">10.1002/(SICI)1097-4571(199210)43:9<628::AID-ASI5>3.0.CO;2-0</pub-id></mixed-citation></ref><ref id="CR56"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Seglen</surname><given-names>PO</given-names></name></person-group><article-title>Why the impact factor of journals should not be used for evaluating research</article-title><source>British Medical Journal</source><year>1997</year><volume>314</volume><fpage>498</fpage><lpage>502</lpage><pub-id pub-id-type="doi">10.1136/bmj.314.7079.497</pub-id><pub-id pub-id-type="pmid">9056804</pub-id></mixed-citation></ref><ref id="CR57"><mixed-citation publication-type="other">Sher, I. H., & Garfield, E. (1965). <italic>New tools for improving and evaluating the effectiveness of research.</italic> Paper presented at the second conference on research program effectiveness, July 27–29, Washington, DC.</mixed-citation></ref><ref id="CR58"><mixed-citation publication-type="book"><person-group person-group-type="author"><name><surname>Sheskin</surname><given-names>DJ</given-names></name></person-group><source>Handbook of parametric and nonparametric statistical procedures</source><year>2011</year><edition>5</edition><publisher-loc>Boca Raton, FL</publisher-loc><publisher-name>Chapman & Hall/CRC</publisher-name></mixed-citation></ref><ref id="CR59"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Small</surname><given-names>H</given-names></name><name><surname>Garfield</surname><given-names>E</given-names></name></person-group><article-title>The geography of science: Disciplinary and national mappings</article-title><source>Journal of information science</source><year>1985</year><volume>11</volume><issue>4</issue><fpage>147</fpage><lpage>159</lpage><pub-id pub-id-type="doi">10.1177/016555158501100402</pub-id></mixed-citation></ref><ref id="CR60"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Small</surname><given-names>H</given-names></name><name><surname>Sweeney</surname><given-names>E</given-names></name></person-group><article-title>Clustering the science citation index using co-citations I. A comparison of methods</article-title><source>Scientometrics</source><year>1985</year><volume>7</volume><fpage>391</fpage><lpage>409</lpage><pub-id pub-id-type="doi">10.1007/BF02017157</pub-id></mixed-citation></ref><ref id="CR62"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vinkler</surname><given-names>P</given-names></name></person-group><article-title>Evaluation of some methods for the relative assessment of scientific publications</article-title><source>Scientometrics</source><year>1986</year><volume>10</volume><issue>3</issue><fpage>157</fpage><lpage>177</lpage><pub-id pub-id-type="doi">10.1007/BF02026039</pub-id></mixed-citation></ref><ref id="CR63"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vinkler</surname><given-names>P</given-names></name></person-group><article-title>The π<sub>v</sub>-index: A new indicator to characterize the impact of journals</article-title><source>Scientometrics</source><year>2010</year><volume>82</volume><issue>3</issue><fpage>461</fpage><lpage>475</lpage><pub-id pub-id-type="doi">10.1007/s11192-010-0182-z</pub-id></mixed-citation></ref><ref id="CR64"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vinkler</surname><given-names>P</given-names></name></person-group><article-title>Application of the distribution of citations among publications in scientometric evaluations</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2011</year><volume>62</volume><issue>10</issue><fpage>1963</fpage><lpage>1978</lpage><pub-id pub-id-type="doi">10.1002/asi.21600</pub-id></mixed-citation></ref><ref id="CR65"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Vinkler</surname><given-names>P</given-names></name></person-group><article-title>The case of scientometricians with the “absolute relative” impact indicator</article-title><source>Journal of Informetrics</source><year>2012</year><volume>6</volume><issue>2</issue><fpage>254</fpage><lpage>264</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2011.12.004</pub-id></mixed-citation></ref><ref id="CR66"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Waltman</surname><given-names>L</given-names></name><name><surname>Eck</surname><given-names>NJ</given-names></name><name><surname>Leeuwen</surname><given-names>TN</given-names></name><name><surname>Visser</surname><given-names>MS</given-names></name><name><surname>Raan</surname><given-names>AFJ</given-names></name></person-group><article-title>Towards a new crown indicator: An empirical analysis</article-title><source>Scientometrics</source><year>2011</year><volume>87</volume><fpage>467</fpage><lpage>481</lpage><pub-id pub-id-type="doi">10.1007/s11192-011-0354-5</pub-id><pub-id pub-id-type="pmid">21654898</pub-id></mixed-citation></ref><ref id="CR67"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Waltman</surname><given-names>L</given-names></name><name><surname>Eck</surname><given-names>NJ</given-names></name><name><surname>Leeuwen</surname><given-names>TN</given-names></name><name><surname>Visser</surname><given-names>MS</given-names></name><name><surname>Raan</surname><given-names>AFJ</given-names></name></person-group><article-title>Towards a new crown indicator: Some theoretical considerations</article-title><source>Journal of Informetrics</source><year>2011</year><volume>5</volume><issue>1</issue><fpage>37</fpage><lpage>47</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2010.08.001</pub-id></mixed-citation></ref><ref id="CR68"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Zhou</surname><given-names>P</given-names></name><name><surname>Leydesdorff</surname><given-names>L</given-names></name></person-group><article-title>Fractional counting of citations in research evaluation: A cross- and interdisciplinary assessment of the Tsinghua University in Beijing</article-title><source>Journal of Informetrics</source><year>2011</year><volume>5</volume><issue>3</issue><fpage>360</fpage><lpage>368</lpage><pub-id pub-id-type="doi">10.1016/j.joi.2011.01.010</pub-id></mixed-citation></ref><ref id="CR69"><mixed-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Zitt</surname><given-names>M</given-names></name><name><surname>Small</surname><given-names>H</given-names></name></person-group><article-title>Modifying the journal impact factor by fractional citation weighting: The audience factor</article-title><source>Journal of the American Society for Information Science and Technology</source><year>2008</year><volume>59</volume><issue>11</issue><fpage>1856</fpage><lpage>1860</lpage><pub-id pub-id-type="doi">10.1002/asi.20880</pub-id></mixed-citation></ref></ref-list><fn-group><fn id="Fn1"><label>1</label><p>Scopus of Elsevier provides similar classification schemes.</p></fn><fn id="Fn2"><label>2</label><p>A similar normalization can be performed at the level of each journal (CPP/JCSm) as originally proposed by Schubert and Braun (<xref ref-type="bibr" rid="CR54">1986</xref>). Vinkler (<xref ref-type="bibr" rid="CR62">1986</xref>) first proposed to extend this expectation to the field level.</p></fn><fn id="Fn3"><label>3</label><p>Second in rank in 2009 was the journal <italic>Nano Today</italic>, with IF = 13.237.</p></fn><fn id="Fn4"><label>4</label><p>The aggregation at the city level presumes further processing the address field using i3cit1.exe, available at <ext-link ext-link-type="uri" xlink:href="http://www.leydesdorff.net/software/i3">http://www.leydesdorff.net/software/i3</ext-link>.</p></fn><fn id="Fn5"><label>5</label><p>Leydesdorff and Bornmann (<xref ref-type="bibr" rid="CR29">2011b</xref>) mention this option, but use the standardized residuals which can also be expected to be Chi-square distributed. Unlike the <italic>z</italic>-test, however, this assumption is not yet mathematically proven in the literature.</p></fn><fn id="Fn6"><label>6</label><p>This error probability increases with the number of pairwise comparisons <italic>c</italic>: <italic>c</italic> = <italic>n</italic> * (<italic>n</italic> − 1)/2. For 50 cases (e.g., journals), the number of pairwise comparisons is 50 * 49/2 = 1,225, and one should therefore test at a significance level of 0.05/1225 = 0.000041 instead of <italic>p</italic> ≤ 0.05.</p></fn><fn id="Fn7"><label>7</label><p>The full title of this journal is: <italic>Materials Science and Engineering A</italic>-<italic>Structural Materials Properties Microstructure and Processing.</italic></p></fn><fn id="Fn8"><label>8</label><p>Alternatively, “fractional counting” attributes addresses proportionally to the number of addresses/paper.</p></fn><fn id="Fn9"><label>9</label><p>The number of addresses is 32,134, and the number of authors: 73,342.</p></fn><fn id="Fn10"><label>10</label><p>The <italic>z</italic>-test is case-based. Bonferroni-Holm correction (Holm <xref ref-type="bibr" rid="CR71">1979</xref>) can then be used for correcting family-wise accumulation of the probability of Type-I errors in multiple comparisons among cases by adjusting the <italic>α</italic>-level (Rüdiger Mutz, personal communication, August 25, 2011). However, the accumulation in this case is not a consequence of multiple comparisons but of summation of I3-values within each case (e.g., city, journal).</p></fn><fn id="Fn11"><label>11</label><p>Leydesdorff and Bornmann (<xref ref-type="bibr" rid="CR29">2011b</xref>) used this expectation, but these authors used the standardized residuals of the Chi-square for the test. The latter test is less conservative and less reliable than the <italic>z</italic>-test.</p></fn><fn id="Fn12"><label>12</label><p>Similarly, the file zperc.txt contains the overlay map when % I3 is tested against % of publications. The score is stored in the field “zperc” within ucities.dbf.</p></fn></fn-group></back></pmc></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Wicri/Belgique/explor/OpenAccessBelV2/Data/Pmc/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000185 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd -nk 000185 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Wicri/Belgique
|area= OpenAccessBelV2
|flux= Pmc
|étape= Corpus
|type= RBID
|clé= PMC:3535360
|texte= An evaluation of impacts in “Nanoscience & nanotechnology”: steps towards standards for citation analysis
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Corpus/RBID.i -Sk "pubmed:23293403" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Corpus/biblio.hfd \
| NlmPubMed2Wicri -a OpenAccessBelV2
| This area was generated with Dilib version V0.6.25. |  |