MersV1, PubMed, Corpus, bibRecord, 001933

BEETL-fastq: a searchable compressed archive for DNA reads.

Identifieur interne : 001933 ( PubMed/Corpus ); précédent : 001932; suivant : 001934

BEETL-fastq: a searchable compressed archive for DNA reads.

Auteurs : Lilian Janin ; Ole Schulz-Trieglaff ; Anthony J. Cox

Source :

Bioinformatics (Oxford, England) [ 1367-4811 ] ; 2014.

RBID : pubmed:24950811

English descriptors

KwdEn :
- Algorithms, Computer Simulation, DNA, Data Compression (methods), Genome, Genome, Human, Genotype, High-Throughput Nucleotide Sequencing, Humans, Neoplasms (genetics), Sequence Analysis, DNA (methods), Software.
MESH :
- chemical : DNA.
- genetics : Neoplasms.
- methods : Data Compression, Sequence Analysis, DNA.
- Algorithms, Computer Simulation, Genome, Genome, Human, Genotype, High-Throughput Nucleotide Sequencing, Humans, Software.

Abstract

FASTQ is a standard file format for DNA sequencing data, which stores both nucleotides and quality scores. A typical sequencing study can easily generate hundreds of gigabytes of FASTQ files, while public archives such as ENA and NCBI and large international collaborations such as the Cancer Genome Atlas can accumulate many terabytes of data in this format. Compression tools such as gzip are often used to reduce the storage burden but have the disadvantage that the data must be decompressed before they can be used. Here, we present BEETL-fastq, a tool that not only compresses FASTQ-formatted DNA reads more compactly than gzip but also permits rapid search for k-mer queries within the archived sequences. Importantly, the full FASTQ record of each matching read or read pair is returned, allowing the search results to be piped directly to any of the many standard tools that accept FASTQ data as input.

DOI: 10.1093/bioinformatics/btu387
PubMed: 24950811

Links to Exploration step

pubmed:24950811

Le document en format XML

<record><TEI><teiHeader><fileDesc><titleStmt><title xml:lang="en">BEETL-fastq: a searchable compressed archive for DNA reads.</title>
<author><name sortKey="Janin, Lilian" sort="Janin, Lilian" uniqKey="Janin L" first="Lilian" last="Janin">Lilian Janin</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
<author><name sortKey="Schulz Trieglaff, Ole" sort="Schulz Trieglaff, Ole" uniqKey="Schulz Trieglaff O" first="Ole" last="Schulz-Trieglaff">Ole Schulz-Trieglaff</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
<author><name sortKey="Cox, Anthony J" sort="Cox, Anthony J" uniqKey="Cox A" first="Anthony J" last="Cox">Anthony J. Cox</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
</titleStmt>
<publicationStmt><idno type="wicri:source">PubMed</idno>
<date when="2014">2014</date>
<idno type="RBID">pubmed:24950811</idno>
<idno type="pmid">24950811</idno>
<idno type="doi">10.1093/bioinformatics/btu387</idno>
<idno type="wicri:Area/PubMed/Corpus">001933</idno>
<idno type="wicri:explorRef" wicri:stream="PubMed" wicri:step="Corpus" wicri:corpus="PubMed">001933</idno>
</publicationStmt>
<sourceDesc><biblStruct><analytic><title xml:lang="en">BEETL-fastq: a searchable compressed archive for DNA reads.</title>
<author><name sortKey="Janin, Lilian" sort="Janin, Lilian" uniqKey="Janin L" first="Lilian" last="Janin">Lilian Janin</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
<author><name sortKey="Schulz Trieglaff, Ole" sort="Schulz Trieglaff, Ole" uniqKey="Schulz Trieglaff O" first="Ole" last="Schulz-Trieglaff">Ole Schulz-Trieglaff</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
<author><name sortKey="Cox, Anthony J" sort="Cox, Anthony J" uniqKey="Cox A" first="Anthony J" last="Cox">Anthony J. Cox</name>
<affiliation><nlm:affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</nlm:affiliation>
</affiliation>
</author>
</analytic>
<series><title level="j">Bioinformatics (Oxford, England)</title>
<idno type="eISSN">1367-4811</idno>
<imprint><date when="2014" type="published">2014</date>
</imprint>
</series>
</biblStruct>
</sourceDesc>
</fileDesc>
<profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Algorithms</term>
<term>Computer Simulation</term>
<term>DNA</term>
<term>Data Compression (methods)</term>
<term>Genome</term>
<term>Genome, Human</term>
<term>Genotype</term>
<term>High-Throughput Nucleotide Sequencing</term>
<term>Humans</term>
<term>Neoplasms (genetics)</term>
<term>Sequence Analysis, DNA (methods)</term>
<term>Software</term>
</keywords>
<keywords scheme="MESH" type="chemical" xml:lang="en"><term>DNA</term>
</keywords>
<keywords scheme="MESH" qualifier="genetics" xml:lang="en"><term>Neoplasms</term>
</keywords>
<keywords scheme="MESH" qualifier="methods" xml:lang="en"><term>Data Compression</term>
<term>Sequence Analysis, DNA</term>
</keywords>
<keywords scheme="MESH" xml:lang="en"><term>Algorithms</term>
<term>Computer Simulation</term>
<term>Genome</term>
<term>Genome, Human</term>
<term>Genotype</term>
<term>High-Throughput Nucleotide Sequencing</term>
<term>Humans</term>
<term>Software</term>
</keywords>
</textClass>
</profileDesc>
</teiHeader>
<front><div type="abstract" xml:lang="en">FASTQ is a standard file format for DNA sequencing data, which stores both nucleotides and quality scores. A typical sequencing study can easily generate hundreds of gigabytes of FASTQ files, while public archives such as ENA and NCBI and large international collaborations such as the Cancer Genome Atlas can accumulate many terabytes of data in this format. Compression tools such as gzip are often used to reduce the storage burden but have the disadvantage that the data must be decompressed before they can be used. Here, we present BEETL-fastq, a tool that not only compresses FASTQ-formatted DNA reads more compactly than gzip but also permits rapid search for k-mer queries within the archived sequences. Importantly, the full FASTQ record of each matching read or read pair is returned, allowing the search results to be piped directly to any of the many standard tools that accept FASTQ data as input.</div>
</front>
</TEI>
<pubmed><MedlineCitation Status="MEDLINE" IndexingMethod="Curated" Owner="NLM"><PMID Version="1">24950811</PMID>
<DateCompleted><Year>2014</Year>
<Month>11</Month>
<Day>21</Day>
</DateCompleted>
<DateRevised><Year>2018</Year>
<Month>12</Month>
<Day>02</Day>
</DateRevised>
<Article PubModel="Print-Electronic"><Journal><ISSN IssnType="Electronic">1367-4811</ISSN>
<JournalIssue CitedMedium="Internet"><Volume>30</Volume>
<Issue>19</Issue>
<PubDate><Year>2014</Year>
<Month>Oct</Month>
</PubDate>
</JournalIssue>
<Title>Bioinformatics (Oxford, England)</Title>
<ISOAbbreviation>Bioinformatics</ISOAbbreviation>
</Journal>
<ArticleTitle>BEETL-fastq: a searchable compressed archive for DNA reads.</ArticleTitle>
<Pagination><MedlinePgn>2796-801</MedlinePgn>
</Pagination>
<ELocationID EIdType="doi" ValidYN="Y">10.1093/bioinformatics/btu387</ELocationID>
<Abstract><AbstractText Label="MOTIVATION" NlmCategory="BACKGROUND">FASTQ is a standard file format for DNA sequencing data, which stores both nucleotides and quality scores. A typical sequencing study can easily generate hundreds of gigabytes of FASTQ files, while public archives such as ENA and NCBI and large international collaborations such as the Cancer Genome Atlas can accumulate many terabytes of data in this format. Compression tools such as gzip are often used to reduce the storage burden but have the disadvantage that the data must be decompressed before they can be used. Here, we present BEETL-fastq, a tool that not only compresses FASTQ-formatted DNA reads more compactly than gzip but also permits rapid search for k-mer queries within the archived sequences. Importantly, the full FASTQ record of each matching read or read pair is returned, allowing the search results to be piped directly to any of the many standard tools that accept FASTQ data as input.</AbstractText>
<AbstractText Label="RESULTS" NlmCategory="RESULTS">We show that 6.6 terabytes of human reads in FASTQ format can be transformed into 1.7 terabytes of indexed files, from where we can search for 1, 10, 100, 1000 and a million of 30-mers in 3, 8, 14, 45 and 567 s, respectively, plus 20 ms per output read. Useful applications of the search capability are highlighted, including the genotyping of structural variant breakpoints and 'in silico pull-down' experiments in which only the reads that cover a region of interest are selectively extracted for the purposes of variant calling or visualization.</AbstractText>
<AbstractText Label="AVAILABILITY AND IMPLEMENTATION" NlmCategory="METHODS">BEETL-fastq is part of the BEETL library, available as a github repository at github.com/BEETL/BEETL.</AbstractText>
<CopyrightInformation>© The Author 2014. Published by Oxford University Press. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com.</CopyrightInformation>
</Abstract>
<AuthorList CompleteYN="Y"><Author ValidYN="Y"><LastName>Janin</LastName>
<ForeName>Lilian</ForeName>
<Initials>L</Initials>
<AffiliationInfo><Affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Schulz-Trieglaff</LastName>
<ForeName>Ole</ForeName>
<Initials>O</Initials>
<AffiliationInfo><Affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</Affiliation>
</AffiliationInfo>
</Author>
<Author ValidYN="Y"><LastName>Cox</LastName>
<ForeName>Anthony J</ForeName>
<Initials>AJ</Initials>
<AffiliationInfo><Affiliation>Computational Biology Group, Illumina Cambridge Ltd., Little Chesterford, Essex CB10 1XL, UK.</Affiliation>
</AffiliationInfo>
</Author>
</AuthorList>
<Language>eng</Language>
<PublicationTypeList><PublicationType UI="D016428">Journal Article</PublicationType>
</PublicationTypeList>
<ArticleDate DateType="Electronic"><Year>2014</Year>
<Month>06</Month>
<Day>20</Day>
</ArticleDate>
</Article>
<MedlineJournalInfo><Country>England</Country>
<MedlineTA>Bioinformatics</MedlineTA>
<NlmUniqueID>9808944</NlmUniqueID>
<ISSNLinking>1367-4803</ISSNLinking>
</MedlineJournalInfo>
<ChemicalList><Chemical><RegistryNumber>9007-49-2</RegistryNumber>
<NameOfSubstance UI="D004247">DNA</NameOfSubstance>
</Chemical>
</ChemicalList>
<CitationSubset>IM</CitationSubset>
<MeshHeadingList><MeshHeading><DescriptorName UI="D000465" MajorTopicYN="N">Algorithms</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D003198" MajorTopicYN="N">Computer Simulation</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D004247" MajorTopicYN="N">DNA</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D044962" MajorTopicYN="N">Data Compression</DescriptorName>
<QualifierName UI="Q000379" MajorTopicYN="Y">methods</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D016678" MajorTopicYN="N">Genome</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D015894" MajorTopicYN="N">Genome, Human</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D005838" MajorTopicYN="N">Genotype</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D059014" MajorTopicYN="N">High-Throughput Nucleotide Sequencing</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D006801" MajorTopicYN="N">Humans</DescriptorName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D009369" MajorTopicYN="N">Neoplasms</DescriptorName>
<QualifierName UI="Q000235" MajorTopicYN="Y">genetics</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D017422" MajorTopicYN="N">Sequence Analysis, DNA</DescriptorName>
<QualifierName UI="Q000379" MajorTopicYN="Y">methods</QualifierName>
</MeshHeading>
<MeshHeading><DescriptorName UI="D012984" MajorTopicYN="N">Software</DescriptorName>
</MeshHeading>
</MeshHeadingList>
</MedlineCitation>
<PubmedData><History><PubMedPubDate PubStatus="entrez"><Year>2014</Year>
<Month>6</Month>
<Day>22</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
<PubMedPubDate PubStatus="pubmed"><Year>2014</Year>
<Month>6</Month>
<Day>22</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
<PubMedPubDate PubStatus="medline"><Year>2014</Year>
<Month>12</Month>
<Day>15</Day>
<Hour>6</Hour>
<Minute>0</Minute>
</PubMedPubDate>
</History>
<PublicationStatus>ppublish</PublicationStatus>
<ArticleIdList><ArticleId IdType="pubmed">24950811</ArticleId>
<ArticleId IdType="pii">btu387</ArticleId>
<ArticleId IdType="doi">10.1093/bioinformatics/btu387</ArticleId>
</ArticleIdList>
</PubmedData>
</pubmed>
</record>

Pour manipuler ce document sous Unix (Dilib)

EXPLOR_STEP=$WICRI_ROOT/Sante/explor/MersV1/Data/PubMed/Corpus

HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 001933 | SxmlIndent | more

HfdSelect -h $EXPLOR_AREA/Data/PubMed/Corpus/biblio.hfd -nk 001933 | SxmlIndent | more

Pour mettre un lien sur cette page dans le réseau Wicri

{{Explor lien
   |wiki=    Sante
   |area=    MersV1
   |flux=    PubMed
   |étape=   Corpus
   |type=    RBID
   |clé=     pubmed:24950811
   |texte=   BEETL-fastq: a searchable compressed archive for DNA reads.
}}

Pour générer des pages wiki

HfdIndexSelect -h $EXPLOR_AREA/Data/PubMed/Corpus/RBID.i   -Sk "pubmed:24950811" \
       | HfdSelect -Kh $EXPLOR_AREA/Data/PubMed/Corpus/biblio.hfd   \
       | NlmPubMed2Wicri -a MersV1

This area was generated with Dilib version V0.6.33.
Data generation: Mon Apr 20 23:26:43 2020. Site generation: Sat Mar 27 09:06:09 2021

	Serveur d'exploration MERS
	Attention, ce site est en cours de développement ! Attention, site généré par des moyens informatiques à partir de corpus bruts. Les informations ne sont donc pas validées.

Serveur d'exploration MERS

BEETL-fastq: a searchable compressed archive for DNA reads.

BEETL-fastq: a searchable compressed archive for DNA reads.

Source :

English descriptors

Abstract

Links to Exploration step

Le document en format XML

Pour manipuler ce document sous Unix (Dilib)

Pour mettre un lien sur cette page dans le réseau Wicri

Pour générer des pages wiki