Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes
Identifieur interne : 000093 ( Pmc/Checkpoint ); précédent : 000092; suivant : 000094Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes
Auteurs : Aksel Paulsen [Danemark] ; S Ren Overgaard [Danemark] ; Jens Martin Lauritsen [Danemark]Source :
- PLoS ONE [ 1932-6203 ] ; 2012.
Abstract
The clinical and scientific usage of patient-reported outcome measures is increasing in the health services. Often paper forms are used. Manual double entry of data is defined as the definitive gold standard for transferring data to an electronic format, but the process is laborious. Automated forms processing may be an alternative, but further validation is warranted.
200 patients were randomly selected from a cohort of 5777 patients who had previously answered two different questionnaires. The questionnaires were scanned using an automated forms processing technique, as well as processed by single and double manual data entry, using the EpiData Entry data entry program. The main outcome measure was the proportion of correctly entered numbers at question, form and study level.
Manual double-key data entry (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) performed better than single-key data entry (error proportion per 1000 fields = 0.370 (95% CI: 0.160–0.729), (p = 0.020)). There was no statistical difference between Optical Mark Recognition (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) and double-key data entry (p = 1.000). With the Intelligent Character Recognition method, there was no statistical difference compared to single-key data entry (error proportion per 1000 fields = 6.734 (95% CI: 0.817–24.113), (p = 0.656)), as well as double-key data entry (error proportion per 1000 fields = 3.367 (95% CI: 0.085–18.616)), (p = 0.319)).
Automated forms processing is a valid alternative to double manual data entry for highly structured forms containing only check boxes, numerical codes and no dates. Automated forms processing can be superior to single manual data entry through a data entry program, depending on the method chosen.
Url:
DOI: 10.1371/journal.pone.0035087
PubMed: 22493733
PubMed Central: 3320865
Affiliations:
Links toward previous steps (curation, corpus...)
Links to Exploration step
PMC:3320865Le document en format XML
<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes</title><author><name sortKey="Paulsen, Aksel" sort="Paulsen, Aksel" uniqKey="Paulsen A" first="Aksel" last="Paulsen">Aksel Paulsen</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><addr-line>Institute of Clinical Research, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Clinical Research, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author><author><name sortKey="Overgaard, S Ren" sort="Overgaard, S Ren" uniqKey="Overgaard S" first="S Ren" last="Overgaard">S Ren Overgaard</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><addr-line>Institute of Clinical Research, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Clinical Research, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author><author><name sortKey="Lauritsen, Jens Martin" sort="Lauritsen, Jens Martin" uniqKey="Lauritsen J" first="Jens Martin" last="Lauritsen">Jens Martin Lauritsen</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff3"><addr-line>Institute of Public Health, Department of Biostatistics, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Public Health, Department of Biostatistics, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">PMC</idno><idno type="pmid">22493733</idno><idno type="pmc">3320865</idno><idno type="url">http://www.ncbi.nlm.nih.gov/pmc/articles/PMC3320865</idno><idno type="RBID">PMC:3320865</idno><idno type="doi">10.1371/journal.pone.0035087</idno><date when="2012">2012</date><idno type="wicri:Area/Pmc/Corpus">000173</idno><idno type="wicri:Area/Pmc/Curation">000173</idno><idno type="wicri:Area/Pmc/Checkpoint">000093</idno></publicationStmt><sourceDesc><biblStruct><analytic><title xml:lang="en" level="a" type="main">Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes</title><author><name sortKey="Paulsen, Aksel" sort="Paulsen, Aksel" uniqKey="Paulsen A" first="Aksel" last="Paulsen">Aksel Paulsen</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><addr-line>Institute of Clinical Research, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Clinical Research, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author><author><name sortKey="Overgaard, S Ren" sort="Overgaard, S Ren" uniqKey="Overgaard S" first="S Ren" last="Overgaard">S Ren Overgaard</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff2"><addr-line>Institute of Clinical Research, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Clinical Research, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author><author><name sortKey="Lauritsen, Jens Martin" sort="Lauritsen, Jens Martin" uniqKey="Lauritsen J" first="Jens Martin" last="Lauritsen">Jens Martin Lauritsen</name><affiliation wicri:level="1"><nlm:aff id="aff1"><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation><affiliation wicri:level="1"><nlm:aff id="aff3"><addr-line>Institute of Public Health, Department of Biostatistics, University of Southern Denmark, Odense, Funen, Denmark</addr-line></nlm:aff><country xml:lang="fr">Danemark</country><wicri:regionArea>Institute of Public Health, Department of Biostatistics, University of Southern Denmark, Odense, Funen</wicri:regionArea><wicri:noRegion>Funen</wicri:noRegion></affiliation></author></analytic><series><title level="j">PLoS ONE</title><idno type="eISSN">1932-6203</idno><imprint><date when="2012">2012</date></imprint></series></biblStruct></sourceDesc></fileDesc><profileDesc><textClass></textClass></profileDesc></teiHeader><front><div type="abstract" xml:lang="en"><sec><title>Background</title><p>The clinical and scientific usage of patient-reported outcome measures is increasing in the health services. Often paper forms are used. Manual double entry of data is defined as the definitive gold standard for transferring data to an electronic format, but the process is laborious. Automated forms processing may be an alternative, but further validation is warranted.</p></sec><sec><title>Methods</title><p>200 patients were randomly selected from a cohort of 5777 patients who had previously answered two different questionnaires. The questionnaires were scanned using an automated forms processing technique, as well as processed by single and double manual data entry, using the EpiData Entry data entry program. The main outcome measure was the proportion of correctly entered numbers at question, form and study level.</p></sec><sec><title>Results</title><p>Manual double-key data entry (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) performed better than single-key data entry (error proportion per 1000 fields = 0.370 (95% CI: 0.160–0.729), (p = 0.020)). There was no statistical difference between Optical Mark Recognition (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) and double-key data entry (p = 1.000). With the Intelligent Character Recognition method, there was no statistical difference compared to single-key data entry (error proportion per 1000 fields = 6.734 (95% CI: 0.817–24.113), (p = 0.656)), as well as double-key data entry (error proportion per 1000 fields = 3.367 (95% CI: 0.085–18.616)), (p = 0.319)).</p></sec><sec><title>Conclusions</title><p>Automated forms processing is a valid alternative to double manual data entry for highly structured forms containing only check boxes, numerical codes and no dates. Automated forms processing can be superior to single manual data entry through a data entry program, depending on the method chosen.</p></sec></div></front><back><div1 type="bibliography"><listBibl><biblStruct><analytic><author><name sortKey="Protti, D" uniqKey="Protti D">D Protti</name></author><author><name sortKey="Wright, G" uniqKey="Wright G">G Wright</name></author><author><name sortKey="Treweek, S" uniqKey="Treweek S">S Treweek</name></author><author><name sortKey="Johansen, I" uniqKey="Johansen I">I Johansen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="O Boyle, Ca" uniqKey="O Boyle C">CA O'Boyle</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lawlor, Da" uniqKey="Lawlor D">DA Lawlor</name></author><author><name sortKey="Chaturvedi, N" uniqKey="Chaturvedi N">N Chaturvedi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ascher, Rn" uniqKey="Ascher R">RN Ascher</name></author><author><name sortKey="Koppelmann, Gm" uniqKey="Koppelmann G">GM Koppelmann</name></author><author><name sortKey="Miller, Mj" uniqKey="Miller M">MJ Miller</name></author><author><name sortKey="Nagy, G" uniqKey="Nagy G">G Nagy</name></author><author><name sortKey="Shelton, Gl" uniqKey="Shelton G">GL Shelton</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nagy, G" uniqKey="Nagy G">G Nagy</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Abele, L" uniqKey="Abele L">L Abele</name></author><author><name sortKey="Wahl, E" uniqKey="Wahl E">E Wahl</name></author><author><name sortKey="Scheri, W" uniqKey="Scheri W">W Scheri</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Toyoda, J" uniqKey="Toyoda J">J Toyoda</name></author><author><name sortKey="Noguchi, Y" uniqKey="Noguchi Y">Y Noguchi</name></author><author><name sortKey="Nishimura, Y" uniqKey="Nishimura Y">Y Nishimura</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wong, Iy" uniqKey="Wong I">IY Wong</name></author><author><name sortKey="Casey, Rg" uniqKey="Casey R">RG Casey</name></author><author><name sortKey="Wahl, Em" uniqKey="Wahl E">EM Wahl</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Casey, Rg" uniqKey="Casey R">RG Casey</name></author><author><name sortKey="Ferguson, Dr" uniqKey="Ferguson D">DR Ferguson</name></author><author><name sortKey="Mohiuddin, Km" uniqKey="Mohiuddin K">KM Mohiuddin</name></author><author><name sortKey="Walach, E" uniqKey="Walach E">E Walach</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ciardiello, G" uniqKey="Ciardiello G">G Ciardiello</name></author><author><name sortKey="Degrandi, Mt" uniqKey="Degrandi M">MT Degrandi</name></author><author><name sortKey="Poccotelli, Mp" uniqKey="Poccotelli M">MP Poccotelli</name></author><author><name sortKey="Scafuro, G" uniqKey="Scafuro G">G Scafuro</name></author><author><name sortKey="Spada, Mr" uniqKey="Spada M">MR Spada</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Esposito, E" uniqKey="Esposito E">E Esposito</name></author><author><name sortKey="Malerba, D" uniqKey="Malerba D">D Malerba</name></author><author><name sortKey="Semerano, G" uniqKey="Semerano G">G Semerano</name></author><author><name sortKey="Annese, E" uniqKey="Annese E">E Annese</name></author><author><name sortKey="Scafuro, G" uniqKey="Scafuro G">G Scafuro</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hagita, N" uniqKey="Hagita N">N Hagita</name></author><author><name sortKey="Masuda, I" uniqKey="Masuda I">I Masuda</name></author><author><name sortKey="Akiyama, T" uniqKey="Akiyama T">T Akiyama</name></author><author><name sortKey="Talahashi, T" uniqKey="Talahashi T">T Talahashi</name></author><author><name sortKey="Naito, S" uniqKey="Naito S">S Naito</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nagy, G" uniqKey="Nagy G">G Nagy</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nakano, Y" uniqKey="Nakano Y">Y Nakano</name></author><author><name sortKey="Fujisawa, H" uniqKey="Fujisawa H">H Fujisawa</name></author><author><name sortKey="Kunisaki, O" uniqKey="Kunisaki O">O Kunisaki</name></author><author><name sortKey="Okada, K" uniqKey="Okada K">K Okada</name></author><author><name sortKey="Hananoi, T" uniqKey="Hananoi T">T Hananoi</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Tsujimoto, S" uniqKey="Tsujimoto S">S Tsujimoto</name></author><author><name sortKey="Asada, H" uniqKey="Asada H">H Asada</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kobak, Ka" uniqKey="Kobak K">KA Kobak</name></author><author><name sortKey="Mundt, Jc" uniqKey="Mundt J">JC Mundt</name></author><author><name sortKey="Greist, Jh" uniqKey="Greist J">JH Greist</name></author><author><name sortKey="Katzelnick, Dj" uniqKey="Katzelnick D">DJ Katzelnick</name></author><author><name sortKey="Jefferson, Jw" uniqKey="Jefferson J">JW Jefferson</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Kubick, Wr" uniqKey="Kubick W">WR Kubick</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Lampe, Aj" uniqKey="Lampe A">AJ Lampe</name></author><author><name sortKey="Weiler, Jm" uniqKey="Weiler J">JM Weiler</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ohmann, C" uniqKey="Ohmann C">C Ohmann</name></author><author><name sortKey="Kuchinke, W" uniqKey="Kuchinke W">W Kuchinke</name></author><author><name sortKey="Canham, S" uniqKey="Canham S">S Canham</name></author><author><name sortKey="Lauritsen, J" uniqKey="Lauritsen J">J Lauritsen</name></author><author><name sortKey="Salas, N" uniqKey="Salas N">N Salas</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rolfson, O" uniqKey="Rolfson O">O Rolfson</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Wahi, Mm" uniqKey="Wahi M">MM Wahi</name></author><author><name sortKey="Parks, Dv" uniqKey="Parks D">DV Parks</name></author><author><name sortKey="Skeate, Rc" uniqKey="Skeate R">RC Skeate</name></author><author><name sortKey="Goldin, Sb" uniqKey="Goldin S">SB Goldin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Goldberg, Si" uniqKey="Goldberg S">SI Goldberg</name></author><author><name sortKey="Niemierko, A" uniqKey="Niemierko A">A Niemierko</name></author><author><name sortKey="Turchin, A" uniqKey="Turchin A">A Turchin</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Weir, Cr" uniqKey="Weir C">CR Weir</name></author><author><name sortKey="Hurdle, Jf" uniqKey="Hurdle J">JF Hurdle</name></author><author><name sortKey="Felgar, Ma" uniqKey="Felgar M">MA Felgar</name></author><author><name sortKey="Hoffman, Jm" uniqKey="Hoffman J">JM Hoffman</name></author><author><name sortKey="Roth, B" uniqKey="Roth B">B Roth</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nahm, Ml" uniqKey="Nahm M">ML Nahm</name></author><author><name sortKey="Pieper, Cf" uniqKey="Pieper C">CF Pieper</name></author><author><name sortKey="Cunningham, Mm" uniqKey="Cunningham M">MM Cunningham</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Paulsen, A" uniqKey="Paulsen A">A Paulsen</name></author><author><name sortKey="Pedersen, Ab" uniqKey="Pedersen A">AB Pedersen</name></author><author><name sortKey="Overgaard, S" uniqKey="Overgaard S">S Overgaard</name></author><author><name sortKey="Roos, Em" uniqKey="Roos E">EM Roos</name></author></analytic></biblStruct><biblStruct></biblStruct><biblStruct><analytic><author><name sortKey="Brooks, R" uniqKey="Brooks R">R Brooks</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Ware, Je" uniqKey="Ware J">JE Ware</name></author><author><name sortKey="Kosinski, M" uniqKey="Kosinski M">M Kosinski</name></author><author><name sortKey="Keller, Sd" uniqKey="Keller S">SD Keller</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Nilsdotter, Ak" uniqKey="Nilsdotter A">AK Nilsdotter</name></author><author><name sortKey="Lohmander, Ls" uniqKey="Lohmander L">LS Lohmander</name></author><author><name sortKey="Klassbo, M" uniqKey="Klassbo M">M Klassbo</name></author><author><name sortKey="Roos, Em" uniqKey="Roos E">EM Roos</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Dawson, J" uniqKey="Dawson J">J Dawson</name></author><author><name sortKey="Fitzpatrick, R" uniqKey="Fitzpatrick R">R Fitzpatrick</name></author><author><name sortKey="Carr, A" uniqKey="Carr A">A Carr</name></author><author><name sortKey="Murray, D" uniqKey="Murray D">D Murray</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Jorgensen, Ck" uniqKey="Jorgensen C">CK Jorgensen</name></author><author><name sortKey="Karlsmose, B" uniqKey="Karlsmose B">B Karlsmose</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Rieder, Hl" uniqKey="Rieder H">HL Rieder</name></author><author><name sortKey="Lauritsen, Jm" uniqKey="Lauritsen J">JM Lauritsen</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Hardin, Jm" uniqKey="Hardin J">JM Hardin</name></author><author><name sortKey="Woodby, Ll" uniqKey="Woodby L">LL Woodby</name></author><author><name sortKey="Crawford, Ma" uniqKey="Crawford M">MA Crawford</name></author><author><name sortKey="Windsor, Ra" uniqKey="Windsor R">RA Windsor</name></author><author><name sortKey="Miller, Tm" uniqKey="Miller T">TM Miller</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Biondich, Pg" uniqKey="Biondich P">PG Biondich</name></author><author><name sortKey="Overhage, Jm" uniqKey="Overhage J">JM Overhage</name></author><author><name sortKey="Dexter, Pr" uniqKey="Dexter P">PR Dexter</name></author><author><name sortKey="Downs, Sm" uniqKey="Downs S">SM Downs</name></author><author><name sortKey="Lemmon, L" uniqKey="Lemmon L">L Lemmon</name></author></analytic></biblStruct><biblStruct><analytic><author><name sortKey="Jinks, C" uniqKey="Jinks C">C Jinks</name></author><author><name sortKey="Jordan, K" uniqKey="Jordan K">K Jordan</name></author><author><name sortKey="Croft, P" uniqKey="Croft P">P Croft</name></author></analytic></biblStruct></listBibl></div1></back></TEI><pmc article-type="research-article"><pmc-dir>properties open_access</pmc-dir>
<front><journal-meta><journal-id journal-id-type="nlm-ta">PLoS One</journal-id><journal-id journal-id-type="iso-abbrev">PLoS ONE</journal-id><journal-id journal-id-type="publisher-id">plos</journal-id><journal-id journal-id-type="pmc">plosone</journal-id><journal-title-group><journal-title>PLoS ONE</journal-title></journal-title-group><issn pub-type="epub">1932-6203</issn><publisher><publisher-name>Public Library of Science</publisher-name><publisher-loc>San Francisco, USA</publisher-loc></publisher></journal-meta><article-meta><article-id pub-id-type="pmid">22493733</article-id><article-id pub-id-type="pmc">3320865</article-id><article-id pub-id-type="publisher-id">PONE-D-11-15903</article-id><article-id pub-id-type="doi">10.1371/journal.pone.0035087</article-id><article-categories><subj-group subj-group-type="heading"><subject>Research Article</subject></subj-group><subj-group subj-group-type="Discipline-v2"><subject>Medicine</subject><subj-group><subject>Clinical Research Design</subject><subj-group><subject>Epidemiology</subject><subject>Survey Research</subject></subj-group></subj-group><subj-group><subject>Epidemiology</subject><subj-group><subject>Clinical Epidemiology</subject><subject>Epidemiological Methods</subject><subject>Survey Methods</subject></subj-group></subj-group><subj-group><subject>Non-Clinical Medicine</subject><subj-group><subject>Health Care Policy</subject><subj-group><subject>Health Systems Strengthening</subject></subj-group></subj-group><subj-group><subject>Health Informatics</subject><subject>Health Services Research</subject></subj-group></subj-group><subj-group><subject>Surgery</subject><subj-group><subject>Orthopedic Surgery</subject><subj-group><subject>Joint Replacement Surgery</subject></subj-group></subj-group></subj-group></subj-group></article-categories><title-group><article-title>Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes</article-title><alt-title alt-title-type="running-head">Data Quality with Automated Forms Processing</alt-title></title-group><contrib-group><contrib contrib-type="author"><name><surname>Paulsen</surname><given-names>Aksel</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="aff" rid="aff2"><sup>2</sup></xref><xref ref-type="corresp" rid="cor1"><sup>*</sup></xref></contrib><contrib contrib-type="author"><name><surname>Overgaard</surname><given-names>Søren</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="aff" rid="aff2"><sup>2</sup></xref></contrib><contrib contrib-type="author"><name><surname>Lauritsen</surname><given-names>Jens Martin</given-names></name><xref ref-type="aff" rid="aff1"><sup>1</sup></xref><xref ref-type="aff" rid="aff3"><sup>3</sup></xref></contrib></contrib-group><aff id="aff1"><label>1</label><addr-line>Department of Orthopaedic Surgery and Traumatology, Odense University Hospital, Odense, Funen, Denmark</addr-line></aff><aff id="aff2"><label>2</label><addr-line>Institute of Clinical Research, University of Southern Denmark, Odense, Funen, Denmark</addr-line></aff><aff id="aff3"><label>3</label><addr-line>Institute of Public Health, Department of Biostatistics, University of Southern Denmark, Odense, Funen, Denmark</addr-line></aff><contrib-group><contrib contrib-type="editor"><name><surname>Boulle</surname><given-names>Andrew</given-names></name><role>Editor</role><xref ref-type="aff" rid="edit1"></xref></contrib></contrib-group><aff id="edit1">Faculty of Health Sciences, University of Cape Town, South Africa</aff><author-notes><corresp id="cor1">* E-mail: <email>Aksel.Paulsen@ouh.regionsyddanmark.dk</email></corresp><fn fn-type="con"><p>Conceived and designed the experiments: AP JML SO. Performed the experiments: AP JML SO. Analyzed the data: AP. Contributed reagents/materials/analysis tools: AP JML SO. Wrote the paper: AP JML SO.</p></fn></author-notes><pub-date pub-type="collection"><year>2012</year></pub-date><pub-date pub-type="epub"><day>6</day><month>4</month><year>2012</year></pub-date><volume>7</volume><issue>4</issue><elocation-id>e35087</elocation-id><history><date date-type="received"><day>16</day><month>8</month><year>2011</year></date><date date-type="accepted"><day>12</day><month>3</month><year>2012</year></date></history><permissions><copyright-statement>Paulsen et al. This is an open-access article distributed under the terms of the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original author and source are credited.</copyright-statement><copyright-year>2012</copyright-year></permissions><abstract><sec><title>Background</title><p>The clinical and scientific usage of patient-reported outcome measures is increasing in the health services. Often paper forms are used. Manual double entry of data is defined as the definitive gold standard for transferring data to an electronic format, but the process is laborious. Automated forms processing may be an alternative, but further validation is warranted.</p></sec><sec><title>Methods</title><p>200 patients were randomly selected from a cohort of 5777 patients who had previously answered two different questionnaires. The questionnaires were scanned using an automated forms processing technique, as well as processed by single and double manual data entry, using the EpiData Entry data entry program. The main outcome measure was the proportion of correctly entered numbers at question, form and study level.</p></sec><sec><title>Results</title><p>Manual double-key data entry (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) performed better than single-key data entry (error proportion per 1000 fields = 0.370 (95% CI: 0.160–0.729), (p = 0.020)). There was no statistical difference between Optical Mark Recognition (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) and double-key data entry (p = 1.000). With the Intelligent Character Recognition method, there was no statistical difference compared to single-key data entry (error proportion per 1000 fields = 6.734 (95% CI: 0.817–24.113), (p = 0.656)), as well as double-key data entry (error proportion per 1000 fields = 3.367 (95% CI: 0.085–18.616)), (p = 0.319)).</p></sec><sec><title>Conclusions</title><p>Automated forms processing is a valid alternative to double manual data entry for highly structured forms containing only check boxes, numerical codes and no dates. Automated forms processing can be superior to single manual data entry through a data entry program, depending on the method chosen.</p></sec></abstract><counts><page-count count="6"></page-count></counts></article-meta></front><body><sec id="s1"><title>Introduction</title><p>Information in the medical services is now almost exclusively based on electronic recording systems in Denmark, including communication between primary and secondary health care systems <xref ref-type="bibr" rid="pone.0035087-Protti1">[1]</xref>. In surgery, among other areas of the health services, there has been a growing focus from medical clinicians on the use of patient-reported outcomes in studies <xref ref-type="bibr" rid="pone.0035087-OBoyle1">[2]</xref>. Internationally, the US Food and Drug Administration has strongly recommended inclusion of patient-recorded outcomes in clinical trials assessing the effect of medical procedures or pharmaceuticals. This has led to a demand for recording larger volumes of information, which traditionally have been collected on paper forms. An alternative to manual data entry has been the introduction of automated reading of such data forms. With an increased focus on measuring and validating measurement tools <xref ref-type="bibr" rid="pone.0035087-Lawlor1">[3]</xref>, it is imperative to assess the quality of automated forms processing and this was the motivation for the current study.</p><p>In the 1960s, research began on document processing <xref ref-type="bibr" rid="pone.0035087-Ascher1">[4]</xref>–<xref ref-type="bibr" rid="pone.0035087-Nagy1">[5]</xref>. With the development of computers and the increasing need to capture large volumes of data, automatic text segmentation and discrimination research gained momentum in the early 1980s <xref ref-type="bibr" rid="pone.0035087-Abele1">[6]</xref>, <xref ref-type="bibr" rid="pone.0035087-Toyoda1">[7]</xref>, <xref ref-type="bibr" rid="pone.0035087-Wong1">[8]</xref>. A variety of data processing systems have been described <xref ref-type="bibr" rid="pone.0035087-Casey1">[9]</xref>, <xref ref-type="bibr" rid="pone.0035087-Ciardiello1">[10]</xref>, <xref ref-type="bibr" rid="pone.0035087-Esposito1">[11]</xref>, <xref ref-type="bibr" rid="pone.0035087-Hagita1">[12]</xref>, <xref ref-type="bibr" rid="pone.0035087-Nagy2">[13]</xref>, <xref ref-type="bibr" rid="pone.0035087-Nakano1">[14]</xref>, <xref ref-type="bibr" rid="pone.0035087-Tsujimoto1">[15]</xref>, among these different kinds of automatic forms processing or scanning procedures <xref ref-type="bibr" rid="pone.0035087-Kobak1">[16]</xref>, <xref ref-type="bibr" rid="pone.0035087-Kubick1">[17]</xref>, <xref ref-type="bibr" rid="pone.0035087-Lampe1">[18]</xref>. A growing commercial industry offers automated forms processing technologies and services. However, manual double entry of data is still defined as the definitive gold standard of good clinical practice <xref ref-type="bibr" rid="pone.0035087-Ohmann1">[19]</xref> for data from collected paper forms, and it has been well-validated <xref ref-type="bibr" rid="pone.0035087-Lampe1">[18]</xref>.</p><p>Internet-based applications for collecting questionnaires instead of using paper forms may be the future, but for now, and in particular when dealing with an elderly population, it is known that some patient groups do not respond adequately to an Internet-based application for collecting patient-reported outcome questionnaires <xref ref-type="bibr" rid="pone.0035087-Rolfson1">[20]</xref>. When data is entered directly via the internet connection, validation is a very complex matter. No other source of information exists to verify correctness of the data since data is only recorded once.</p><p>Automated forms processing technologies are advocated mainly because of potential data quality improvement and likely time and cost reductions. Manual double-key entering of data by key punching is laborious and can be costly. Transcription of data from paper forms into an electronic database can be a nontrivial source of error <xref ref-type="bibr" rid="pone.0035087-Wahi1">[21]</xref>. Both manual key entering and direct text entry may result in a serious reduction in data quality, if the proportion of erroneous entries is large, as seen in some clinical research databases <xref ref-type="bibr" rid="pone.0035087-Goldberg1">[22]</xref>, <xref ref-type="bibr" rid="pone.0035087-Weir1">[23]</xref>.</p><p>Automated forms processing is a method by which one can ‘automatically’ capture information entered into data fields by scanning, and converting it into an electronic format. The data is captured from particular zones and stored in an electronic format. This input method can automate data processing by using pre-defined templates and configurations. A template in this case, would be a map of the document, detailing where the data fields are located within the form. Most of the data are recognised automatically using the pre-specified data characteristics, but if the program is uncertain, verification by a human operator is required.</p><p>There are different technologies of automated forms processing. Optical Mark Recognition [OMR] is the least expensive solution but can only be used for recognition of check/mark boxes on a form. The more advanced Intelligent Character Recognition [ICR] can be used for recognition of machine-printed and handwritten characters. In this project, we have used ICR to recognise hand-printed characters, and OMR to identify check boxes filled in by hand on printed forms.</p><p>There have been few reports on the quality of automated forms processing and usage in medical settings, relatively few data collection systems are well-described with respect to data quality <xref ref-type="bibr" rid="pone.0035087-Nahm1">[24]</xref>, and further research on automated forms processing performance in this setting is therefore warranted.</p><p>The aim is therefore to examine and validate an up-to-date automated forms processing system, by comparing paper-based and scanned patient-reported outcome forms with single and double manually entered data.</p></sec><sec sec-type="methods" id="s2"><title>Methods</title><sec id="s2a"><title>Ethics</title><p>The study was approved by the Danish National Board of Health and the Danish Data Protection Agency (journal number 2008-41-2593). The Science Ethics Committee of the Region of Southern Denmark rejected registration since this is a registry based study without collection of biological data. The study was carried out in accordance with the World Medical Association’s Declaration of Helsinki, and all patients gave informed written consent to participate. None of the authors have existing or potential competing interests.</p><fig id="pone-0035087-g001" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0035087.g001</object-id><label>Figure 1</label><caption><title>Questionnaire Pairs (200 patients).</title></caption><graphic xlink:href="pone.0035087.g001"></graphic></fig></sec><sec id="s2b"><title>Design</title><p>The study was based on a larger study with a cohort of 5777 patients from the Danish Hip Arthroplasty Registry <xref ref-type="bibr" rid="pone.0035087-Paulsen1">[25]</xref>. The cohort consisted of patients over 18 years of age, with primary total hip arthroplasty, regardless of diagnosis, who underwent an operation1–2, 5–6, and 10–11 years earlier. Every patient had received two different patient-reported outcome questionnaires, one general and one disease-specific. The following questionnaires were included in the study: the EuroQoL-5D-3L [EQ-5D], (consisting of the EQ-VAS and the EQ-5D Index) <xref ref-type="bibr" rid="pone.0035087-Group1">[26]</xref>, <xref ref-type="bibr" rid="pone.0035087-Brooks1">[27]</xref>, SF-12 Health Survey [SF-12] (yielding MCS and PCS) <xref ref-type="bibr" rid="pone.0035087-WareJE1">[28]</xref>, Hip dysfunction and Osteoarthritis Outcome Score [HOOS] (consisting of HOOS Pain, HOOS PS, and HOOS QoL) <xref ref-type="bibr" rid="pone.0035087-Nilsdotter1">[29]</xref>, and Oxford 12-item Hip Score [OHS] <xref ref-type="bibr" rid="pone.0035087-Dawson1">[30]</xref>. From the total cohort 200 patients were randomly selected in four groups (blocks) of 50 patients for each year. None of the groups received the same pair of questionnaires so as to maximise the potential for statistical comparison, see <xref ref-type="fig" rid="pone-0035087-g001">Figure 1</xref>. Patient characteristics are listed in <xref ref-type="table" rid="pone-0035087-t001">Table 1</xref>. We used paper forms to administer our questionnaires, and postal administration to deliver them.</p><p>Sample size and power calculations: Based on 297 available EQ-VAS items, the comparison of methods, assuming an error proportion of 1% by double-entry, had 80% power to detect a 4% higher error proportion by ICR.</p><table-wrap id="pone-0035087-t001" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0035087.t001</object-id><label>Table 1</label><caption><title>Patient characteristics.</title></caption><alternatives><graphic id="pone-0035087-t001-1" xlink:href="pone.0035087.t001"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col></colgroup><thead><tr><td align="left" rowspan="1" colspan="1">Category</td><td align="left" rowspan="1" colspan="1">All patients</td><td align="left" rowspan="1" colspan="1">Group 1</td><td align="left" rowspan="1" colspan="1">Group 2</td><td align="left" rowspan="1" colspan="1">Group 3</td><td align="left" rowspan="1" colspan="1">Group 4</td></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Population (n)</td><td align="left" rowspan="1" colspan="1">n = 200</td><td align="left" rowspan="1" colspan="1">n = 50</td><td align="left" rowspan="1" colspan="1">n = 50</td><td align="left" rowspan="1" colspan="1">n = 50</td><td align="left" rowspan="1" colspan="1">n = 50</td></tr><tr><td align="left" rowspan="1" colspan="1">Percent of total</td><td align="left" rowspan="1" colspan="1">100</td><td align="left" rowspan="1" colspan="1">25</td><td align="left" rowspan="1" colspan="1">25</td><td align="left" rowspan="1" colspan="1">25</td><td align="left" rowspan="1" colspan="1">25</td></tr><tr><td align="left" rowspan="1" colspan="1">Age <xref ref-type="table-fn" rid="nt101">*</xref> (median)</td><td align="left" rowspan="1" colspan="1">72</td><td align="left" rowspan="1" colspan="1">72</td><td align="left" rowspan="1" colspan="1">71</td><td align="left" rowspan="1" colspan="1">70</td><td align="left" rowspan="1" colspan="1">74</td></tr><tr><td align="left" rowspan="1" colspan="1">Range (years)</td><td align="left" rowspan="1" colspan="1">25–95</td><td align="left" rowspan="1" colspan="1">47–90</td><td align="left" rowspan="1" colspan="1">25–90</td><td align="left" rowspan="1" colspan="1">44–95</td><td align="left" rowspan="1" colspan="1">34–94</td></tr><tr><td align="left" rowspan="1" colspan="1">Sex: Female (%)</td><td align="left" rowspan="1" colspan="1">N = 118 (59%)</td><td align="left" rowspan="1" colspan="1">N = 29 (58%)</td><td align="left" rowspan="1" colspan="1">N = 29 (58%)</td><td align="left" rowspan="1" colspan="1">N = 32 (64%)</td><td align="left" rowspan="1" colspan="1">N = 28 (56%)</td></tr></tbody></table></alternatives><table-wrap-foot><fn id="nt101"><label>*</label><p>Age of patients on date of questionnaire dispatch in years.</p></fn></table-wrap-foot></table-wrap></sec><sec id="s2c"><title>Items and Forms</title><p>A form was defined as a questionnaire, an item as a single question on a questionnaire and a data field as a possible answer category for an item. EQ-5D contains 6 items in total, 5 single items plus the EQ-VAS. OHS and SF-12 each contains 12 items, and HOOS contains 19 items. The EQ-5D instruction states “By placing a tick in one box in each of the five groups below, please indicate which statements best describe your own health state today”. There are three possible answers (and thus three check boxes resulting in three data fields the scanner is coded to read) for each EQ-5D single item, e.g. for the item ‘Mobility’ categories are: “I have no problems in walking about”, “I have some problems in walking about” and “I am confined to bed”. EQ-VAS requires the respondent to indicate on a thermometer scale from 0 (‘worst imaginable’) to 100 (‘best imaginable’), how good or bad the responder’s health is on that particular day by drawing a line from a box to the appropriate point on the scale which indicates how good or bad his/her health state is on the day. All items from OHS, HOOS, SF-12 and the 5 items on EQ-5D, could be read from the checkboxes by OMR, and only the EQ-VAS had to be read by ICR. The latter was done from a field where the respondent wrote the scale value from 0–100.</p><p>Scanning setup was an up-to-date automated forms processing system. The scanner was a Kodak i640 scanner (Kodak Canada Inc., Toronto, Ontario), scanning in 200 DPI, at a speed of 83 pages per minute. Scanning was done in TIFF format, which is approved in Danish law. OCR for AnyDoc ®, version 5.012e (AnyDoc Software Inc., Tampa, Florida) was used for questionnaire setup, and processing. For verifying, AnyDoc®VERIFYIt version 5.0 (AnyDoc Software Inc., Tampa, Florida) was used. HP Elitebook 8530p computers (Hewlett-Packard Company, Palo Alta, California), with Windows version XP and the Microsoft 2003 packages (Microsoft Corporation, Redmond, Washington) were used. Before the study started, we performed template testing of the questionnaires to make sure the setup of the template and placement of the data fields were optimal. The scanner was regularly calibrated. Prior to the scanning, extensive manuals with decision rules for all questionnaires, as well as codebooks were produced to account for any uncertainty. Standard format layout was taken from each questionnaire included with minimal layout adjustments to optimise automated forms processing readability.</p></sec><sec id="s2d"><title>Manual Validation During Scanning</title><p>Manual validation was conducted when the automated forms processing system could not convert an answer due to poor or ambiguous questionnaire completion. In these circumstances the scanner stops, and cannot scan further until a human operator manually validates the correct code for the questionnaire answer in question.</p></sec><sec id="s2e"><title>Manual Data Entry</title><p>A combined structured questionnaire for all the forms used, including limiting definitions for entry of out of range values was defined using EpiData Data Entry software (EpiData Association, <ext-link ext-link-type="uri" xlink:href="Http://www.epidata.dk">Http://www.epidata.dk</ext-link>). EpiData Entry was also used for the double-key data entry and the program control of the data entry.</p></sec><sec id="s2f"><title>Comparison</title><p>To compare the results from automated forms processing with single- and double-key punching, the data were compared using all three methods with EpiData Entry by direct comparisons. The data were also checked for missing values, invalid values and out of range values in STATA, and by reference to the original questionnaires. All manual validations were recorded. A correct data entry was defined when the automated forms processing, single-key punching and double-key punching gave the same data at field (variable) level. In case of differences, we manually consulted the original questionnaire twice, and found the responder’s answer in accordance with the manuals for handling the questionnaires, as well as the individual coding guidance books.</p><table-wrap id="pone-0035087-t002" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0035087.t002</object-id><label>Table 2</label><caption><title>Number of Questionnaires, Items and Data fields in relation to Processing Method and Questionnaires.</title></caption><alternatives><graphic id="pone-0035087-t002-2" xlink:href="pone.0035087.t002"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col></colgroup><thead><tr><td align="left" rowspan="1" colspan="1">Questionnaire/Scanning method</td><td align="left" rowspan="1" colspan="1">Total number of Questionnaires</td><td align="left" rowspan="1" colspan="1">Total number of Items</td><td align="left" rowspan="1" colspan="1">Total number of Data fields</td></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">EQ-5D</td><td align="left" rowspan="1" colspan="1">99</td><td align="left" rowspan="1" colspan="1">594</td><td align="left" rowspan="1" colspan="1">1782</td></tr><tr><td align="left" rowspan="1" colspan="1">SF-12</td><td align="left" rowspan="1" colspan="1">100</td><td align="left" rowspan="1" colspan="1">1200</td><td align="left" rowspan="1" colspan="1">4700</td></tr><tr><td align="left" rowspan="1" colspan="1">HOOS</td><td align="left" rowspan="1" colspan="1">99</td><td align="left" rowspan="1" colspan="1">1881</td><td align="left" rowspan="1" colspan="1">9405</td></tr><tr><td align="left" rowspan="1" colspan="1">OHS</td><td align="left" rowspan="1" colspan="1">100</td><td align="left" rowspan="1" colspan="1">1200</td><td align="left" rowspan="1" colspan="1">6000</td></tr><tr><td align="left" rowspan="1" colspan="1">Total</td><td align="left" rowspan="1" colspan="1">398</td><td align="left" rowspan="1" colspan="1">4875</td><td align="left" rowspan="1" colspan="1">21887</td></tr><tr><td align="left" rowspan="1" colspan="1">Scanned with ICR</td><td align="left" rowspan="1" colspan="1">99</td><td align="left" rowspan="1" colspan="1">99 <xref ref-type="table-fn" rid="nt102">*</xref></td><td align="left" rowspan="1" colspan="1">297 <xref ref-type="table-fn" rid="nt103">†</xref></td></tr></tbody></table></alternatives><table-wrap-foot><fn id="nt102"><label>*</label><p>1 per EQ-5D questionnaire.</p></fn><fn id="nt103"><label>†</label><p>Up to 3 digits per item.</p></fn></table-wrap-foot></table-wrap></sec><sec id="s2g"><title>Statistical Methods</title><p>We studied the error proportion overall, for each of the four different questionnaires, and for each individual patient and tabulated this in subgroups by sex and age groups (<60 years, and >60 years) with binomial confidence intervals. Group difference was tested with a Chi Square test. Error proportions were calculated as proportion of errors per 1000 data field with binomial exact 95% confidence intervals (95% CI) (STATA procedure cii). Validation of the automated forms processing in relation to person ID, was done in comparison with the original sample of all patients (n = 5777), with STATA assert command. Descriptive statistics were used to describe patient characteristics. The STATA software Version 10.1 and 11.0 (StataCorp LP, Texas, USA) were used for all statistical analyses. Due to the pre-specified and low number of tests, we saw no reason to adjust the p-level by multicomparison principles.</p></sec></sec><sec id="s3"><title>Results</title><p>The numbers of questionnaires, items and data fields are listed in <xref ref-type="table" rid="pone-0035087-t002">Table 2</xref>. For ICR (<xref ref-type="table" rid="pone-0035087-t003">Table 3</xref>) there was no statistically significant difference between double-key entering (error proportion per 1000 fields = 3.367 (95% CI: 0.085–18.616)) and single-key entering (error proportion per 1000 fields = 6.734 (95% CI: 0.817–24.113), (p = 0.565)), no statistical difference between automated forms processing (error proportion per 1000 fields = 10.101 (95% CI: 2.088–29.234)) and double-key entering (p = 0.319), nor any statistical difference between automated forms processing and single-key entering (p = 0.656). For OMR (<xref ref-type="table" rid="pone-0035087-t004">Table 4</xref>), automated forms processing (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) performed better than single-key entering (error proportion per 1000 fields = 0.370 (95% CI: 0.160–0.729), (p = 0.020)), double-key entering (error proportion per 1000 fields = 0.046 (95% CI: 0.001–0.258)) performed better than single-key entering (p = 0.020), and automated forms processing and double-key entering performed equally (p = 1.000).</p><table-wrap id="pone-0035087-t003" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0035087.t003</object-id><label>Table 3</label><caption><title>Errors using Intelligent Character Recognition (ICR).</title></caption><alternatives><graphic id="pone-0035087-t003-3" xlink:href="pone.0035087.t003"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col></colgroup><thead><tr><td align="left" rowspan="1" colspan="1">Category</td><td align="left" rowspan="1" colspan="1">Single-key entered data</td><td align="left" rowspan="1" colspan="1">Double-key entered data</td><td align="left" rowspan="1" colspan="1">Automated forms processing</td></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Number of errors</td><td align="left" rowspan="1" colspan="1">2 errors</td><td align="left" rowspan="1" colspan="1">1 error</td><td align="left" rowspan="1" colspan="1">3 errors</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Questionnaire (n = 99)</td><td align="left" rowspan="1" colspan="1">2.02% (0.25–7.11)</td><td align="left" rowspan="1" colspan="1">1.01% (0.03–5.50)</td><td align="left" rowspan="1" colspan="1">3.03% (0.63–8.60)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Item (n = 99)</td><td align="left" rowspan="1" colspan="1">2.02% (0.25–7.11)</td><td align="left" rowspan="1" colspan="1">1.01% (0.03–5.50)</td><td align="left" rowspan="1" colspan="1">3.03% (0.63–8.60)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Data field (n = 297)</td><td align="left" rowspan="1" colspan="1">0.67% (0.08–2.41)</td><td align="left" rowspan="1" colspan="1">0.34% (0.01–1.86)</td><td align="left" rowspan="1" colspan="1">1.01% (0.21–2.92)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/10000 Data fields</td><td align="left" rowspan="1" colspan="1">67.34</td><td align="left" rowspan="1" colspan="1">33.67</td><td align="left" rowspan="1" colspan="1">101.01</td></tr></tbody></table></alternatives><table-wrap-foot><fn id="nt104"><label></label><p>n = number scanned, (95% Binomial Confidence Interval).</p></fn></table-wrap-foot></table-wrap><table-wrap id="pone-0035087-t004" position="float"><object-id pub-id-type="doi">10.1371/journal.pone.0035087.t004</object-id><label>Table 4</label><caption><title>Errors using Optic mark Recognition (OMR).</title></caption><alternatives><graphic id="pone-0035087-t004-4" xlink:href="pone.0035087.t004"></graphic><table frame="hsides" rules="groups"><colgroup span="1"><col align="left" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col><col align="center" span="1"></col></colgroup><thead><tr><td align="left" rowspan="1" colspan="1">Category</td><td align="left" rowspan="1" colspan="1">Single-key entered data</td><td align="left" rowspan="1" colspan="1">Double-key entered data</td><td align="left" rowspan="1" colspan="1">Automated forms processing</td></tr></thead><tbody><tr><td align="left" rowspan="1" colspan="1">Number of errors</td><td align="left" rowspan="1" colspan="1">8 errors</td><td align="left" rowspan="1" colspan="1">1 error</td><td align="left" rowspan="1" colspan="1">1 error</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Questionnaire (n = 398)</td><td align="left" rowspan="1" colspan="1">2.01% (0.87–3.92)</td><td align="left" rowspan="1" colspan="1">0.25% (0.01–1.39)</td><td align="left" rowspan="1" colspan="1">0.25% (0.01–1.39)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Item (n = 4776)</td><td align="left" rowspan="1" colspan="1">0.17% (0.07–0.33)</td><td align="left" rowspan="1" colspan="1">0.02% (0.00–0.12)</td><td align="left" rowspan="1" colspan="1">0.02% (0.00–0.12)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/Data field (n = 21608)</td><td align="left" rowspan="1" colspan="1">0.04% (0.02–0.07)</td><td align="left" rowspan="1" colspan="1">0.00% (0.00–0.03)</td><td align="left" rowspan="1" colspan="1">0.00% (0.00–0.03)</td></tr><tr><td align="left" rowspan="1" colspan="1">Errors/10000 Data fields</td><td align="left" rowspan="1" colspan="1">3.70</td><td align="left" rowspan="1" colspan="1">0.46</td><td align="left" rowspan="1" colspan="1">0.46</td></tr></tbody></table></alternatives><table-wrap-foot><fn id="nt105"><label></label><p>n = number scanned, (95% Binomial Confidence Interval).</p></fn></table-wrap-foot></table-wrap><p>We found no difference in performance for the different questionnaires with the automated forms processing in OMR (p = 0.609), with double-key entering (p = 0.644), or single-key entering (p = 0.148). Concerning gender, we found no statistical differences for ICR (p = 0.304, p = 0.239, p = 0.095), or OMR (p = 0.409, p = 0.409, p = 0.371). Similarly, there were no differences concerning age for ICR (p = 0.520, p = 0.711, p = 0.711), or OMR (p = 0.687, p = 0.687, p = 0.904).</p><p>There were substantial differences in the percentage of manually validated items between the questionnaires and automated forms processing methods: 0.25% (OHS), 0.41% (HOOS Pain), 0.51% (HOOS QoL), 0.61% (HOOS PS), 1.42% (SF-12 PCS and MCS), 2.22% (EQ-5D Index) and 20.20% (EQ-VAS). These differences were statistically significant (p<0.001).</p></sec><sec id="s4"><title>Discussion</title><sec id="s4a"><title>Summary</title><p>We found an extremely low level of error with automated forms processing using OMR. It performed the same as double-key entering and performed better than single-key entering. We found an error level of 0.46 per 10,000 data fields read (OMR), which is better than earlier reports <xref ref-type="bibr" rid="pone.0035087-Jorgensen1">[31]</xref>.</p><p>Concerning ICR, we found an error level per data field of up to one percent in single-key entered data, double-key entered data, and automated forms processing. Only one item (EQ-VAS) required ICR, and therefore relatively few data fields could be included in the ICR analyses, which must be taken into consideration in the interpretation. A very high proportion of items required manual validation on EQ-VAS compared with the other questionnaires, and we will argue that this is because of ICR. It is clearly more difficult for the AFP system to identify a hand-printed character (number) correctly than to identify if a check box is marked, also suggested by the higher number of errors per 10000 data fields in ICR compared to OMR. There are many different ICR systems available, and we cannot rule out that a different ICR system might give better results. Further improvements in ICR technology could possibly decrease the error level to the level of OMR, but this has to be examined in future studies.</p></sec><sec id="s4b"><title>Challenges with Data Quality of Questionnaires</title><p>There are many potential errors from questionnaire data. The table from the work of Reider and Lauritsen <xref ref-type="bibr" rid="pone.0035087-Rieder1">[32]</xref> conceptualizes the potential errors arising from data capture, poor design of the data entry form, no program constraints on data entry, single-entry manual key punching and lack of validation in these studies. Automated forms processing has the potential to remove some of these pitfalls, and potentially improve data quality from questionnaires. Relying on internet based data entry could result in an error level comparable to single manual data entry, but the validity of internet based solutions warrants further research in particular in relation to possible age and or subgroup differences potentially resulting in information bias.</p><p>Cost: Further studies should assess the cost of modern automated forms processing systems. Earlier reports have shown processing time was reduced to about one half to one third of that of manual data entry and wage expenses were reduced to about one third to one quarter, but found that a very large number of forms needed to be processed in order to recover the considerable initial investment <xref ref-type="bibr" rid="pone.0035087-Jorgensen1">[31]</xref>. Even though the cost of equipment for automated forms processing data capture has decreased considerably in recent decades, substantial time and computer expertise is still required for implementation.</p><p>We believe our study is representative of a wide variety of research and clinical settings where paper form questionnaires are used. Total hip arthroplasty is indicated for patients with pain and functional disabilities or reduced quality of life. The population is an extensively studied elderly population, with a mean age in Denmark of 70/67 years (female/male), the patients have a spectrum of comorbid conditions and they constitute a suitable and interesting population in relation to validation of automated forms processing.</p></sec><sec id="s4c"><title>Benefits</title><p>There are several potential benefits of using automated forms processing, including a low error level, an improved data verification process and, more importantly, (especially in big studies with many respondents) a significant reduction in time required for data entry <xref ref-type="bibr" rid="pone.0035087-Hardin1">[33]</xref>. In a registry setting, it is important to achieve an efficient data collection procedure. Some studies find automated forms processing approximately three times as fast as the standard method of data entry, with a digit recognition rate of 92.4% <xref ref-type="bibr" rid="pone.0035087-Biondich1">[34]</xref>. The quality of automated forms processing has been found in earlier reports to be acceptable, and studies report a data entry error of as low as 0.041% for all questionnaire items <xref ref-type="bibr" rid="pone.0035087-Jinks1">[35]</xref>. Automated forms processing was validated in Denmark in 1998, and was then found to perform slightly better than single data entry, but worse than double data entry <xref ref-type="bibr" rid="pone.0035087-Jorgensen1">[31]</xref>. However, in the last 12 years, there has been a rapid development in both software and hardware, and we have found that an up-to-date system can perform as well as double-key manual entry.</p></sec><sec id="s4d"><title>Conclusion</title><p>Automated forms processing can yield excellent results provided use of highly structured questionnaires. OMR performed equally as well as manual double-key entering, and better than single-key entering. Regarding ICR, we cannot draw firm conclusions due to the limited data available in this study, and therefore further research, as well as improvement in ICR technology, is warranted.</p></sec></sec></body><back><fn-group><fn fn-type="conflict"><p><bold>Competing Interests: </bold>The authors have declared that no competing interests exist.</p></fn><fn fn-type="financial-disclosure"><p><bold>Funding: </bold>This study was supported by Region of Southern Denmark [095382] (<ext-link ext-link-type="uri" xlink:href="http://www.regionsyddanmark.dk/wm228983">http://www.regionsyddanmark.dk/wm228983</ext-link>), The Danish Rheumatism Association [R63-A830-B373, R71-A1043, R69-A1427, R80-A1282-B373] (<ext-link ext-link-type="uri" xlink:href="http://www.gigtforeningen.dk">http://www.gigtforeningen.dk</ext-link>), and Odense University Hospital [2-41-4-00065-2010](<ext-link ext-link-type="uri" xlink:href="http://www.ouh.dk/wm259883">http://www.ouh.dk/wm259883</ext-link>). The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript.</p></fn></fn-group><ref-list><title>References</title><ref id="pone.0035087-Protti1"><label>1</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Protti</surname><given-names>D</given-names></name><name><surname>Wright</surname><given-names>G</given-names></name><name><surname>Treweek</surname><given-names>S</given-names></name><name><surname>Johansen</surname><given-names>I</given-names></name></person-group><year>2006</year><article-title>Primary care computing in England and Scotland: a comparison with Denmark.</article-title><source>InformPrimCare</source><volume>14</volume><fpage>93</fpage><lpage>99</lpage></element-citation></ref><ref id="pone.0035087-OBoyle1"><label>2</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>O'Boyle</surname><given-names>CA</given-names></name></person-group><year>1992</year><article-title>Assessment of quality of life in surgery.</article-title><source>British Journal of Surgery</source><volume>79</volume><fpage>395</fpage><lpage>398</lpage><pub-id pub-id-type="pmid">1596716</pub-id></element-citation></ref><ref id="pone.0035087-Lawlor1"><label>3</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Lawlor</surname><given-names>DA</given-names></name><name><surname>Chaturvedi</surname><given-names>N</given-names></name></person-group><year>2010</year><article-title>Methods of measurements in epidemiology -call for a new type of paper in the IJE.</article-title><source>International Journal of Epidemiology</source><volume>39</volume><fpage>1133</fpage><lpage>1136</lpage></element-citation></ref><ref id="pone.0035087-Ascher1"><label>4</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ascher</surname><given-names>RN</given-names></name><name><surname>Koppelmann</surname><given-names>GM</given-names></name><name><surname>Miller</surname><given-names>MJ</given-names></name><name><surname>Nagy</surname><given-names>G</given-names></name><name><surname>Shelton</surname><given-names>GL</given-names><suffix>Jr</suffix></name></person-group><year>1971</year><article-title>An interactive system for reading unformatted printed text.</article-title><source>IEEE TransComput</source><volume>C-20</volume><fpage>1527</fpage><lpage>1543</lpage></element-citation></ref><ref id="pone.0035087-Nagy1"><label>5</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nagy</surname><given-names>G</given-names></name></person-group><year>1968</year><article-title>A preliminary investigation of techniques for the automated reading of unformatted text.</article-title><source>CommACM</source><volume>11</volume><fpage>480</fpage><lpage>487</lpage></element-citation></ref><ref id="pone.0035087-Abele1"><label>6</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Abele</surname><given-names>L</given-names></name><name><surname>Wahl</surname><given-names>E</given-names></name><name><surname>Scheri</surname><given-names>W</given-names></name></person-group><year>1981</year><article-title>Procedures for an automatic segmentation of text graphic and halftone regions in document.</article-title><publisher-name>Proc2nd Scandinavian Confon Image Analysis</publisher-name><fpage>177</fpage><lpage>182</lpage></element-citation></ref><ref id="pone.0035087-Toyoda1"><label>7</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Toyoda</surname><given-names>J</given-names></name><name><surname>Noguchi</surname><given-names>Y</given-names></name><name><surname>Nishimura</surname><given-names>Y</given-names></name></person-group><year>1982</year><article-title>Study of extracting Japanese newspaper article.</article-title><publisher-name>Proc6th IntConfon Pattern Recognition</publisher-name><fpage>1113</fpage><lpage>1115</lpage></element-citation></ref><ref id="pone.0035087-Wong1"><label>8</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wong</surname><given-names>IY</given-names></name><name><surname>Casey</surname><given-names>RG</given-names></name><name><surname>Wahl</surname><given-names>EM</given-names></name></person-group><year>1982</year><article-title>Document analysis system.</article-title><source>IBM JReasearch Develop</source><volume>26</volume><fpage>647</fpage><lpage>656</lpage></element-citation></ref><ref id="pone.0035087-Casey1"><label>9</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Casey</surname><given-names>RG</given-names></name><name><surname>Ferguson</surname><given-names>DR</given-names></name><name><surname>Mohiuddin</surname><given-names>KM</given-names></name><name><surname>Walach</surname><given-names>E</given-names></name></person-group><year>1992</year><article-title>An intelligent forms processing system.</article-title><source>Machine VisAppl</source><volume>5</volume><fpage>143</fpage><lpage>155</lpage></element-citation></ref><ref id="pone.0035087-Ciardiello1"><label>10</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Ciardiello</surname><given-names>G</given-names></name><name><surname>Degrandi</surname><given-names>MT</given-names></name><name><surname>Poccotelli</surname><given-names>MP</given-names></name><name><surname>Scafuro</surname><given-names>G</given-names></name><name><surname>Spada</surname><given-names>MR</given-names></name></person-group><year>1988</year><article-title>An experimental system for office document handling and text recognition.</article-title><publisher-name>Proc9th IntConfon Pattern Recognition</publisher-name><fpage>739</fpage><lpage>743</lpage></element-citation></ref><ref id="pone.0035087-Esposito1"><label>11</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Esposito</surname><given-names>E</given-names></name><name><surname>Malerba</surname><given-names>D</given-names></name><name><surname>Semerano</surname><given-names>G</given-names></name><name><surname>Annese</surname><given-names>E</given-names></name><name><surname>Scafuro</surname><given-names>G</given-names></name></person-group><year>1990</year><article-title>An experimental page layout recognition system for office document automatic classification: An integrated approach for inductive generalization.</article-title><publisher-name>Proc10th IntConfon Pattern Recognition</publisher-name><fpage>557</fpage><lpage>562</lpage></element-citation></ref><ref id="pone.0035087-Hagita1"><label>12</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hagita</surname><given-names>N</given-names></name><name><surname>Masuda</surname><given-names>I</given-names></name><name><surname>Akiyama</surname><given-names>T</given-names></name><name><surname>Talahashi</surname><given-names>T</given-names></name><name><surname>Naito</surname><given-names>S</given-names></name></person-group><year>1985</year><article-title>Approach to smart document reader system.</article-title><source>ProcCVPR̀</source><volume>85</volume><fpage>550</fpage><lpage>557</lpage></element-citation></ref><ref id="pone.0035087-Nagy2"><label>13</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nagy</surname><given-names>G</given-names></name></person-group><year>1990</year><article-title>Towards a structured-document-image utility.</article-title><volume>ProcSSPR90</volume><fpage>293</fpage><lpage>309</lpage></element-citation></ref><ref id="pone.0035087-Nakano1"><label>14</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Nakano</surname><given-names>Y</given-names></name><name><surname>Fujisawa</surname><given-names>H</given-names></name><name><surname>Kunisaki</surname><given-names>O</given-names></name><name><surname>Okada</surname><given-names>K</given-names></name><name><surname>Hananoi</surname><given-names>T</given-names></name></person-group><year>1986</year><article-title>A document understanding system incorporating with character recognition.</article-title><publisher-name>Proc8th IntConfon Pattern Recognition</publisher-name><fpage>801</fpage><lpage>803</lpage></element-citation></ref><ref id="pone.0035087-Tsujimoto1"><label>15</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Tsujimoto</surname><given-names>S</given-names></name><name><surname>Asada</surname><given-names>H</given-names></name></person-group><year>1990</year><article-title>Understanding multiarticled documents.</article-title><publisher-name>Proc10th IntConfon Pattern Recognition</publisher-name><fpage>551</fpage><lpage>556</lpage></element-citation></ref><ref id="pone.0035087-Kobak1"><label>16</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kobak</surname><given-names>KA</given-names></name><name><surname>Mundt</surname><given-names>JC</given-names></name><name><surname>Greist</surname><given-names>JH</given-names></name><name><surname>Katzelnick</surname><given-names>DJ</given-names></name><name><surname>Jefferson</surname><given-names>JW</given-names></name></person-group><year>2000</year><article-title>Computer assessment of depression: automating the Hamilton Depression rating scale.</article-title><publisher-name>Drug Information Journal</publisher-name><fpage>145</fpage><lpage>156</lpage></element-citation></ref><ref id="pone.0035087-Kubick1"><label>17</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Kubick</surname><given-names>WR</given-names></name></person-group><year>1998</year><article-title>The elegant machine: applying technology to optimize the clinical trial.</article-title><publisher-name>Drug Information Journal</publisher-name><fpage>861</fpage><lpage>869</lpage></element-citation></ref><ref id="pone.0035087-Lampe1"><label>18</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Lampe</surname><given-names>AJ</given-names></name><name><surname>Weiler</surname><given-names>JM</given-names></name></person-group><year>1988</year><article-title>Data capture from the sponsors and investigators' perspectives: balancing quality, speed, and cost.</article-title><publisher-name>DrugInf J</publisher-name><fpage>871</fpage><lpage>886</lpage></element-citation></ref><ref id="pone.0035087-Ohmann1"><label>19</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ohmann</surname><given-names>C</given-names></name><name><surname>Kuchinke</surname><given-names>W</given-names></name><name><surname>Canham</surname><given-names>S</given-names></name><name><surname>Lauritsen</surname><given-names>J</given-names></name><name><surname>Salas</surname><given-names>N</given-names></name><etal></etal></person-group><year>2011</year><article-title>Standard requirements for GCP-compliant data management in multinational clinical trials.</article-title><source>Trials</source><volume>12</volume><fpage>85</fpage><pub-id pub-id-type="pmid">21426576</pub-id></element-citation></ref><ref id="pone.0035087-Rolfson1"><label>20</label><element-citation publication-type="other"><person-group person-group-type="author"><name><surname>Rolfson</surname><given-names>O</given-names></name></person-group><year>2010</year><source>Patient-reported Outcome Measures and Health-economic Aspects of Total Hip Arthroplasty -A study of the Swedish Hip Arthroplasty Register: Institute of Clinical Sciences at Sahlgrenska Academy, University of Gothenburg</source></element-citation></ref><ref id="pone.0035087-Wahi1"><label>21</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Wahi</surname><given-names>MM</given-names></name><name><surname>Parks</surname><given-names>DV</given-names></name><name><surname>Skeate</surname><given-names>RC</given-names></name><name><surname>Goldin</surname><given-names>SB</given-names></name></person-group><year>2008</year><article-title>Reducing errors from the electronic transcription of data collected on paper forms: a research data case study.</article-title><source>J AmMedInformAssoc</source><volume>15</volume><fpage>386</fpage><lpage>389</lpage></element-citation></ref><ref id="pone.0035087-Goldberg1"><label>22</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Goldberg</surname><given-names>SI</given-names></name><name><surname>Niemierko</surname><given-names>A</given-names></name><name><surname>Turchin</surname><given-names>A</given-names></name></person-group><year>2008</year><article-title>Analysis of data errors in clinical research databases.</article-title><publisher-name>AMIAAnnuSympProc</publisher-name><fpage>242</fpage><lpage>246</lpage></element-citation></ref><ref id="pone.0035087-Weir1"><label>23</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Weir</surname><given-names>CR</given-names></name><name><surname>Hurdle</surname><given-names>JF</given-names></name><name><surname>Felgar</surname><given-names>MA</given-names></name><name><surname>Hoffman</surname><given-names>JM</given-names></name><name><surname>Roth</surname><given-names>B</given-names></name><etal></etal></person-group><year>2003</year><article-title>Direct text entry in electronic progress notes. An evaluation of input errors.</article-title><source>Methods Inf Med</source><volume>42</volume><fpage>61</fpage><lpage>67</lpage><pub-id pub-id-type="pmid">12695797</pub-id></element-citation></ref><ref id="pone.0035087-Nahm1"><label>24</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nahm</surname><given-names>ML</given-names></name><name><surname>Pieper</surname><given-names>CF</given-names></name><name><surname>Cunningham</surname><given-names>MM</given-names></name></person-group><year>2008</year><article-title>Quantifying data quality for clinical trials using electronic data capture.</article-title><source>PLoSOne</source><volume>3</volume><fpage>e3049</fpage></element-citation></ref><ref id="pone.0035087-Paulsen1"><label>25</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Paulsen</surname><given-names>A</given-names></name><name><surname>Pedersen</surname><given-names>AB</given-names></name><name><surname>Overgaard</surname><given-names>S</given-names></name><name><surname>Roos</surname><given-names>EM</given-names></name></person-group><year>2012</year><article-title>Feasibility of four patient-reported outcome measures in a registry setting.</article-title><publisher-name>A cross-sectional study of 6000 patients from the Danish Hip Arthroplasty Registry. Acta Orthop In press</publisher-name></element-citation></ref><ref id="pone.0035087-Group1"><label>26</label><element-citation publication-type="journal"><collab>Group TE</collab><year>1990</year><article-title>EuroQol–a new facility for the measurement of health-related quality of life. The EuroQol Group.</article-title><source>Health Policy</source><volume>16</volume><fpage>199</fpage><lpage>208</lpage><pub-id pub-id-type="pmid">10109801</pub-id></element-citation></ref><ref id="pone.0035087-Brooks1"><label>27</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Brooks</surname><given-names>R</given-names></name></person-group><year>1996</year><article-title>EuroQol: the current state of play.</article-title><source>Health Policy</source><volume>37</volume><fpage>53</fpage><lpage>72</lpage><pub-id pub-id-type="pmid">10158943</pub-id></element-citation></ref><ref id="pone.0035087-WareJE1"><label>28</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Ware</surname><given-names>JE</given-names><suffix>Jr</suffix></name><name><surname>Kosinski</surname><given-names>M</given-names></name><name><surname>Keller</surname><given-names>SD</given-names></name></person-group><year>1996</year><article-title>A 12-Item Short-Form Health Survey: construction of scales and preliminary tests of reliability and validity.</article-title><source>Medical Care</source><volume>34</volume><fpage>220</fpage><lpage>233</lpage><pub-id pub-id-type="pmid">8628042</pub-id></element-citation></ref><ref id="pone.0035087-Nilsdotter1"><label>29</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Nilsdotter</surname><given-names>AK</given-names></name><name><surname>Lohmander</surname><given-names>LS</given-names></name><name><surname>Klassbo</surname><given-names>M</given-names></name><name><surname>Roos</surname><given-names>EM</given-names></name></person-group><year>2003</year><article-title>Hip disability and osteoarthritis outcome score (HOOS)–validity and responsiveness in total hip replacement.</article-title><source>BMCMusculoskeletDisord</source><volume>4</volume><fpage>10</fpage></element-citation></ref><ref id="pone.0035087-Dawson1"><label>30</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Dawson</surname><given-names>J</given-names></name><name><surname>Fitzpatrick</surname><given-names>R</given-names></name><name><surname>Carr</surname><given-names>A</given-names></name><name><surname>Murray</surname><given-names>D</given-names></name></person-group><year>1996</year><article-title>Questionnaire on the perceptions of patients about total hip replacement.</article-title><source>Journal of Bone and Joint SurgeryBritish Volume</source><volume>78</volume><fpage>185</fpage><lpage>190</lpage></element-citation></ref><ref id="pone.0035087-Jorgensen1"><label>31</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Jorgensen</surname><given-names>CK</given-names></name><name><surname>Karlsmose</surname><given-names>B</given-names></name></person-group><year>1998</year><article-title>Validation of automated forms processing A comparison of TeleformTM with manual data entry.</article-title><source>Computers in Biology and Medicine</source><volume>28</volume><fpage>659</fpage><lpage>667</lpage><pub-id pub-id-type="pmid">9878977</pub-id></element-citation></ref><ref id="pone.0035087-Rieder1"><label>32</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Rieder</surname><given-names>HL</given-names></name><name><surname>Lauritsen</surname><given-names>JM</given-names></name></person-group><year>2011</year><article-title>Quality assurance of data: ensuring that numbers reflect operational definitions and contain real measurements.</article-title><source>IntJ TubercLung Dis</source><volume>15</volume><fpage>296</fpage><lpage>304</lpage></element-citation></ref><ref id="pone.0035087-Hardin1"><label>33</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Hardin</surname><given-names>JM</given-names></name><name><surname>Woodby</surname><given-names>LL</given-names></name><name><surname>Crawford</surname><given-names>MA</given-names></name><name><surname>Windsor</surname><given-names>RA</given-names></name><name><surname>Miller</surname><given-names>TM</given-names></name></person-group><year>2005</year><article-title>Data collection in a multisite project: Teleform.</article-title><source>Public Health Nursing</source><volume>22</volume><fpage>366</fpage><lpage>370</lpage><pub-id pub-id-type="pmid">16150019</pub-id></element-citation></ref><ref id="pone.0035087-Biondich1"><label>34</label><element-citation publication-type="book"><person-group person-group-type="author"><name><surname>Biondich</surname><given-names>PG</given-names></name><name><surname>Overhage</surname><given-names>JM</given-names></name><name><surname>Dexter</surname><given-names>PR</given-names></name><name><surname>Downs</surname><given-names>SM</given-names></name><name><surname>Lemmon</surname><given-names>L</given-names></name><etal></etal></person-group><year>2002</year><article-title>A modern optical character recognition system in a real world clinical setting: some accuracy and feasibility observations.</article-title><publisher-name>ProcAMIASymp</publisher-name><fpage>56</fpage><lpage>60</lpage></element-citation></ref><ref id="pone.0035087-Jinks1"><label>35</label><element-citation publication-type="journal"><person-group person-group-type="author"><name><surname>Jinks</surname><given-names>C</given-names></name><name><surname>Jordan</surname><given-names>K</given-names></name><name><surname>Croft</surname><given-names>P</given-names></name></person-group><year>2003</year><article-title>Evaluation of a computer-assisted data entry procedure (including Teleform) for large-scale mailed surveys.</article-title><source>Computers in Biology and Medicine</source><volume>33</volume><fpage>425</fpage><lpage>437</lpage><pub-id pub-id-type="pmid">12860466</pub-id></element-citation></ref></ref-list></back></pmc><affiliations><list><country><li>Danemark</li></country></list><tree><country name="Danemark"><noRegion><name sortKey="Paulsen, Aksel" sort="Paulsen, Aksel" uniqKey="Paulsen A" first="Aksel" last="Paulsen">Aksel Paulsen</name></noRegion><name sortKey="Lauritsen, Jens Martin" sort="Lauritsen, Jens Martin" uniqKey="Lauritsen J" first="Jens Martin" last="Lauritsen">Jens Martin Lauritsen</name><name sortKey="Lauritsen, Jens Martin" sort="Lauritsen, Jens Martin" uniqKey="Lauritsen J" first="Jens Martin" last="Lauritsen">Jens Martin Lauritsen</name><name sortKey="Overgaard, S Ren" sort="Overgaard, S Ren" uniqKey="Overgaard S" first="S Ren" last="Overgaard">S Ren Overgaard</name><name sortKey="Overgaard, S Ren" sort="Overgaard, S Ren" uniqKey="Overgaard S" first="S Ren" last="Overgaard">S Ren Overgaard</name><name sortKey="Paulsen, Aksel" sort="Paulsen, Aksel" uniqKey="Paulsen A" first="Aksel" last="Paulsen">Aksel Paulsen</name></country></tree></affiliations></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/OcrV1/Data/Pmc/Checkpoint
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000093 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd -nk 000093 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= OcrV1
|flux= Pmc
|étape= Checkpoint
|type= RBID
|clé= PMC:3320865
|texte= Quality of Data Entry Using Single Entry, Double Entry and Automated Forms Processing–An Example Based on a Study of Patient-Reported Outcomes
}}
Pour générer des pages wiki
HfdIndexSelect -h $EXPLOR_AREA/Data/Pmc/Checkpoint/RBID.i -Sk "pubmed:22493733" \
| HfdSelect -Kh $EXPLOR_AREA/Data/Pmc/Checkpoint/biblio.hfd \
| NlmPubMed2Wicri -a OcrV1
| This area was generated with Dilib version V0.6.32. |  |