Scientific workflow management and the Kepler system
Identifieur interne : 000526 ( Istex/Corpus ); précédent : 000525; suivant : 000527Scientific workflow management and the Kepler system
Auteurs : Bertram Lud Scher ; Ilkay Altintas ; Chad Berkley ; Dan Higgins ; Efrat Jaeger ; Matthew Jones ; Edward A. Lee ; Jing Tao ; Yang ZhaoSource :
- Concurrency and Computation: Practice and Experience [ 1532-0626 ] ; 2006-08-25.
English descriptors
- KwdEn :
Abstract
Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call scientific workflows. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.
Url:
DOI: 10.1002/cpe.994
Links to Exploration step
ISTEX:30E3B595D35F4B9A8B4605A53FA65B4A123D59F9Le document en format XML
<record><TEI wicri:istexFullTextTei="biblStruct"><teiHeader><fileDesc><titleStmt><title xml:lang="en">Scientific workflow management and the Kepler system</title><author><name sortKey="Lud Scher, Bertram" sort="Lud Scher, Bertram" uniqKey="Lud Scher B" first="Bertram" last="Lud Scher">Bertram Lud Scher</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation><affiliation><mods:affiliation>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Altintas, Ilkay" sort="Altintas, Ilkay" uniqKey="Altintas I" first="Ilkay" last="Altintas">Ilkay Altintas</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Berkley, Chad" sort="Berkley, Chad" uniqKey="Berkley C" first="Chad" last="Berkley">Chad Berkley</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Higgins, Dan" sort="Higgins, Dan" uniqKey="Higgins D" first="Dan" last="Higgins">Dan Higgins</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Jaeger, Efrat" sort="Jaeger, Efrat" uniqKey="Jaeger E" first="Efrat" last="Jaeger">Efrat Jaeger</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Jones, Matthew" sort="Jones, Matthew" uniqKey="Jones M" first="Matthew" last="Jones">Matthew Jones</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Lee, Edward A" sort="Lee, Edward A" uniqKey="Lee E" first="Edward A." last="Lee">Edward A. Lee</name><affiliation><mods:affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Tao, Jing" sort="Tao, Jing" uniqKey="Tao J" first="Jing" last="Tao">Jing Tao</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Zhao, Yang" sort="Zhao, Yang" uniqKey="Zhao Y" first="Yang" last="Zhao">Yang Zhao</name><affiliation><mods:affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</mods:affiliation></affiliation></author></titleStmt><publicationStmt><idno type="wicri:source">ISTEX</idno><idno type="RBID">ISTEX:30E3B595D35F4B9A8B4605A53FA65B4A123D59F9</idno><date when="2006" year="2006">2006</date><idno type="doi">10.1002/cpe.994</idno><idno type="url">https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/fulltext/pdf</idno><idno type="wicri:Area/Istex/Corpus">000526</idno></publicationStmt><sourceDesc><biblStruct><analytic><title level="a" type="main" xml:lang="en">Scientific workflow management and the Kepler system</title><author><name sortKey="Lud Scher, Bertram" sort="Lud Scher, Bertram" uniqKey="Lud Scher B" first="Bertram" last="Lud Scher">Bertram Lud Scher</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation><affiliation><mods:affiliation>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Altintas, Ilkay" sort="Altintas, Ilkay" uniqKey="Altintas I" first="Ilkay" last="Altintas">Ilkay Altintas</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Berkley, Chad" sort="Berkley, Chad" uniqKey="Berkley C" first="Chad" last="Berkley">Chad Berkley</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Higgins, Dan" sort="Higgins, Dan" uniqKey="Higgins D" first="Dan" last="Higgins">Dan Higgins</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Jaeger, Efrat" sort="Jaeger, Efrat" uniqKey="Jaeger E" first="Efrat" last="Jaeger">Efrat Jaeger</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Jones, Matthew" sort="Jones, Matthew" uniqKey="Jones M" first="Matthew" last="Jones">Matthew Jones</name><affiliation><mods:affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Lee, Edward A" sort="Lee, Edward A" uniqKey="Lee E" first="Edward A." last="Lee">Edward A. Lee</name><affiliation><mods:affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Tao, Jing" sort="Tao, Jing" uniqKey="Tao J" first="Jing" last="Tao">Jing Tao</name><affiliation><mods:affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</mods:affiliation></affiliation></author><author><name sortKey="Zhao, Yang" sort="Zhao, Yang" uniqKey="Zhao Y" first="Yang" last="Zhao">Yang Zhao</name><affiliation><mods:affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</mods:affiliation></affiliation></author></analytic><monogr></monogr><series><title level="j">Concurrency and Computation: Practice and Experience</title><title level="j" type="abbrev">Concurrency Computat.: Pract. Exper.</title><idno type="ISSN">1532-0626</idno><idno type="eISSN">1532-0634</idno><imprint><publisher>John Wiley & Sons, Ltd.</publisher><pubPlace>Chichester, UK</pubPlace><date type="published" when="2006-08-25">2006-08-25</date><biblScope unit="volume">18</biblScope><biblScope unit="issue">10</biblScope><biblScope unit="page" from="1039">1039</biblScope><biblScope unit="page" to="1065">1065</biblScope></imprint><idno type="ISSN">1532-0626</idno></series><idno type="istex">30E3B595D35F4B9A8B4605A53FA65B4A123D59F9</idno><idno type="DOI">10.1002/cpe.994</idno><idno type="ArticleID">CPE994</idno></biblStruct></sourceDesc><seriesStmt><idno type="ISSN">1532-0626</idno></seriesStmt></fileDesc><profileDesc><textClass><keywords scheme="KwdEn" xml:lang="en"><term>Grid workflows</term><term>dataflow networks</term><term>problem‐solving environments</term><term>scientific data management</term><term>scientific workflows</term></keywords></textClass><langUsage><language ident="en">en</language></langUsage></profileDesc></teiHeader><front><div type="abstract" xml:lang="en">Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call scientific workflows. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.</div></front></TEI><istex><corpusName>wiley</corpusName><author><json:item><name>Bertram Ludäscher</name><affiliations><json:string>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</json:string><json:string>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</json:string></affiliations></json:item><json:item><name>Ilkay Altintas</name><affiliations><json:string>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</json:string></affiliations></json:item><json:item><name>Chad Berkley</name><affiliations><json:string>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</json:string></affiliations></json:item><json:item><name>Dan Higgins</name><affiliations><json:string>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</json:string></affiliations></json:item><json:item><name>Efrat Jaeger</name><affiliations><json:string>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</json:string></affiliations></json:item><json:item><name>Matthew Jones</name><affiliations><json:string>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</json:string></affiliations></json:item><json:item><name>Edward A. Lee</name><affiliations><json:string>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</json:string></affiliations></json:item><json:item><name>Jing Tao</name><affiliations><json:string>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</json:string></affiliations></json:item><json:item><name>Yang Zhao</name><affiliations><json:string>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</json:string></affiliations></json:item></author><subject><json:item><lang><json:string>eng</json:string></lang><value>scientific workflows</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>Grid workflows</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>scientific data management</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>problem‐solving environments</value></json:item><json:item><lang><json:string>eng</json:string></lang><value>dataflow networks</value></json:item></subject><articleId><json:string>CPE994</json:string></articleId><language><json:string>eng</json:string></language><originalGenre><json:string>article</json:string></originalGenre><abstract>Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call scientific workflows. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.</abstract><qualityIndicators><score>7.628</score><pdfVersion>1.3</pdfVersion><pdfPageSize>567 x 737 pts</pdfPageSize><refBibsNative>true</refBibsNative><keywordCount>5</keywordCount><abstractCharCount>1475</abstractCharCount><pdfWordCount>11352</pdfWordCount><pdfCharCount>69405</pdfCharCount><pdfPageCount>27</pdfPageCount><abstractWordCount>219</abstractWordCount></qualityIndicators><title>Scientific workflow management and the Kepler system</title><genre><json:string>article</json:string></genre><host><volume>18</volume><publisherId><json:string>CPE</json:string></publisherId><pages><total>27</total><last>1065</last><first>1039</first></pages><issn><json:string>1532-0626</json:string></issn><issue>10</issue><author><json:item><name>Geoffrey C. Fox</name></json:item><json:item><name>Dennis Gannon</name></json:item></author><subject><json:item><value>Research Article</value></json:item></subject><genre><json:string>journal</json:string></genre><language><json:string>unknown</json:string></language><eissn><json:string>1532-0634</json:string></eissn><title>Concurrency and Computation: Practice and Experience</title><doi><json:string>10.1002/(ISSN)1532-0634</json:string></doi></host><publicationDate>2006</publicationDate><copyrightDate>2006</copyrightDate><doi><json:string>10.1002/cpe.994</json:string></doi><id>30E3B595D35F4B9A8B4605A53FA65B4A123D59F9</id><score>0.13969265</score><fulltext><json:item><original>true</original><mimetype>application/pdf</mimetype><extension>pdf</extension><uri>https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/fulltext/pdf</uri></json:item><json:item><original>false</original><mimetype>application/zip</mimetype><extension>zip</extension><uri>https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/fulltext/zip</uri></json:item><istex:fulltextTEI uri="https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/fulltext/tei"><teiHeader><fileDesc><titleStmt><title level="a" type="main" xml:lang="en">Scientific workflow management and the Kepler system</title></titleStmt><publicationStmt><authority>ISTEX</authority><publisher>John Wiley & Sons, Ltd.</publisher><pubPlace>Chichester, UK</pubPlace><availability><p>Copyright © 2005 John Wiley & Sons, Ltd.</p></availability><date>2006</date></publicationStmt><notesStmt><note>NSF/ITR - No. 0225676 (SEEK); No. CCR‐00225610 (Chess); No. 0225673 (GEON); No. 0325963 (ROADNet);</note><note>DOE SciDAC - No. DE‐FC02‐01ER25486 (SDM);</note><note>NIH/NCRR - No. 1R24 RR019701‐01;</note><note>Biomedical Informatics Research Network Coordinating Center (BIRN‐CC)</note><note>NSF/DBI - No. 0078296 (Resurgence);</note></notesStmt><sourceDesc><biblStruct type="inbook"><analytic><title level="a" type="main" xml:lang="en">Scientific workflow management and the Kepler system</title><author xml:id="author-1"><persName><forename type="first">Bertram</forename><surname>Ludäscher</surname></persName><note type="correspondence"><p>Correspondence: Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</p></note><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation><affiliation>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</affiliation></author><author xml:id="author-2"><persName><forename type="first">Ilkay</forename><surname>Altintas</surname></persName><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation></author><author xml:id="author-3"><persName><forename type="first">Chad</forename><surname>Berkley</surname></persName><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation></author><author xml:id="author-4"><persName><forename type="first">Dan</forename><surname>Higgins</surname></persName><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation></author><author xml:id="author-5"><persName><forename type="first">Efrat</forename><surname>Jaeger</surname></persName><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation></author><author xml:id="author-6"><persName><forename type="first">Matthew</forename><surname>Jones</surname></persName><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation></author><author xml:id="author-7"><persName><forename type="first">Edward A.</forename><surname>Lee</surname></persName><affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</affiliation></author><author xml:id="author-8"><persName><forename type="first">Jing</forename><surname>Tao</surname></persName><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation></author><author xml:id="author-9"><persName><forename type="first">Yang</forename><surname>Zhao</surname></persName><affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</affiliation></author></analytic><monogr><title level="j">Concurrency and Computation: Practice and Experience</title><title level="j" type="abbrev">Concurrency Computat.: Pract. Exper.</title><idno type="pISSN">1532-0626</idno><idno type="eISSN">1532-0634</idno><idno type="DOI">10.1002/(ISSN)1532-0634</idno><imprint><publisher>John Wiley & Sons, Ltd.</publisher><pubPlace>Chichester, UK</pubPlace><date type="published" when="2006-08-25"></date><biblScope unit="volume">18</biblScope><biblScope unit="issue">10</biblScope><biblScope unit="page" from="1039">1039</biblScope><biblScope unit="page" to="1065">1065</biblScope></imprint></monogr><idno type="istex">30E3B595D35F4B9A8B4605A53FA65B4A123D59F9</idno><idno type="DOI">10.1002/cpe.994</idno><idno type="ArticleID">CPE994</idno></biblStruct></sourceDesc></fileDesc><profileDesc><creation><date>2006</date></creation><langUsage><language ident="en">en</language></langUsage><abstract xml:lang="en"><p>Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call scientific workflows. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.</p></abstract><textClass xml:lang="en"><keywords scheme="keyword"><list><head>keywords</head><item><term>scientific workflows</term></item><item><term>Grid workflows</term></item><item><term>scientific data management</term></item><item><term>problem‐solving environments</term></item><item><term>dataflow networks</term></item></list></keywords></textClass><textClass><keywords scheme="Journal Subject"><list><head>article-category</head><item><term>Research Article</term></item></list></keywords></textClass></profileDesc><revisionDesc><change when="2004-06-01">Received</change><change when="2005-04-27">Registration</change><change when="2006-08-25">Published</change></revisionDesc></teiHeader></istex:fulltextTEI><json:item><original>false</original><mimetype>text/plain</mimetype><extension>txt</extension><uri>https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/fulltext/txt</uri></json:item></fulltext><metadata><istex:metadataXml wicri:clean="Wiley, elements deleted: body"><istex:xmlDeclaration>version="1.0" encoding="UTF-8" standalone="yes"</istex:xmlDeclaration><istex:document><component version="2.0" type="serialArticle" xml:lang="en"><header><publicationMeta level="product"><publisherInfo><publisherName>John Wiley & Sons, Ltd.</publisherName><publisherLoc>Chichester, UK</publisherLoc></publisherInfo><doi registered="yes">10.1002/(ISSN)1532-0634</doi><issn type="print">1532-0626</issn><issn type="electronic">1532-0634</issn><idGroup><id type="product" value="CPE"></id></idGroup><titleGroup><title type="main" xml:lang="en" sort="CONCURRENCY AND COMPUTATION: PRACTICE AND EXPERIENCE">Concurrency and Computation: Practice and Experience</title><title type="short">Concurrency Computat.: Pract. Exper.</title></titleGroup><selfCitationGroup><citation type="ancestor" xml:id="cit1"><journalTitle>Concurrency: Practice and Experience</journalTitle><accessionId ref="info:x-wiley/issn/10403108">1040-3108</accessionId><accessionId ref="info:x-wiley/issn/10969128">1096-9128</accessionId><pubYear year="2000">2000</pubYear><vol>12</vol><issue>15</issue></citation></selfCitationGroup></publicationMeta><publicationMeta level="part" position="100"><doi origin="wiley" registered="yes">10.1002/cpe.v18:10</doi><titleGroup><title type="specialIssueTitle">Workflow in Grid Systems</title></titleGroup><numberingGroup><numbering type="journalVolume" number="18">18</numbering><numbering type="journalIssue">10</numbering></numberingGroup><creators><creator xml:id="sped1" creatorRole="sponsoringEditor"><personName><givenNames>Geoffrey C.</givenNames><familyName>Fox</familyName></personName></creator><creator xml:id="sped2" creatorRole="sponsoringEditor"><personName><givenNames>Dennis</givenNames><familyName>Gannon</familyName></personName></creator></creators><coverDate startDate="2006-08-25">25 August 2006</coverDate></publicationMeta><publicationMeta level="unit" type="article" position="30" status="forIssue"><doi origin="wiley" registered="yes">10.1002/cpe.994</doi><idGroup><id type="unit" value="CPE994"></id></idGroup><countGroup><count type="pageTotal" number="27"></count></countGroup><titleGroup><title type="articleCategory">Research Article</title><title type="tocHeading1">Research Articles</title></titleGroup><copyright ownership="publisher">Copyright © 2005 John Wiley & Sons, Ltd.</copyright><eventGroup><event type="manuscriptReceived" date="2004-06-01"></event><event type="manuscriptRevised" date="2005-04-06"></event><event type="manuscriptAccepted" date="2005-04-27"></event><event type="publishedOnlineEarlyUnpaginated" date="2005-12-13"></event><event type="firstOnline" date="2005-12-13"></event><event type="publishedOnlineFinalForm" date="2006-07-19"></event><event type="xmlConverted" agent="Converter:JWSART34_TO_WML3G version:2.3.3 mode:FullText source:HeaderRef result:HeaderRef" date="2010-03-19"></event><event type="xmlConverted" agent="Converter:WILEY_ML3G_TO_WILEY_ML3GV2 version:3.8.8" date="2014-01-16"></event><event type="xmlConverted" agent="Converter:WML3G_To_WML3G version:4.1.7 mode:FullText,remove_FC" date="2014-10-17"></event></eventGroup><numberingGroup><numbering type="pageFirst">1039</numbering><numbering type="pageLast">1065</numbering></numberingGroup><correspondenceTo>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</correspondenceTo><linkGroup><link type="toTypesetVersion" href="file:CPE.CPE994.pdf"></link></linkGroup></publicationMeta><contentMeta><countGroup><count type="figureTotal" number="9"></count><count type="tableTotal" number="0"></count><count type="referenceTotal" number="71"></count></countGroup><titleGroup><title type="main" xml:lang="en">Scientific workflow management and the Kepler system</title><title type="short" xml:lang="en">SCIENTIFIC WORKFLOW MANAGEMENT AND THE KEPLER SYSTEM</title></titleGroup><creators><creator xml:id="au1" creatorRole="author" affiliationRef="#af1 #af2" corresponding="yes"><personName><givenNames>Bertram</givenNames><familyName>Ludäscher</familyName></personName><contactDetails><email>ludaesch@ucdavis.edu</email></contactDetails></creator><creator xml:id="au2" creatorRole="author" affiliationRef="#af1"><personName><givenNames>Ilkay</givenNames><familyName>Altintas</familyName></personName></creator><creator xml:id="au3" creatorRole="author" affiliationRef="#af3"><personName><givenNames>Chad</givenNames><familyName>Berkley</familyName></personName></creator><creator xml:id="au4" creatorRole="author" affiliationRef="#af3"><personName><givenNames>Dan</givenNames><familyName>Higgins</familyName></personName></creator><creator xml:id="au5" creatorRole="author" affiliationRef="#af1"><personName><givenNames>Efrat</givenNames><familyName>Jaeger</familyName></personName></creator><creator xml:id="au6" creatorRole="author" affiliationRef="#af3"><personName><givenNames>Matthew</givenNames><familyName>Jones</familyName></personName></creator><creator xml:id="au7" creatorRole="author" affiliationRef="#af4"><personName><givenNames>Edward A.</givenNames><familyName>Lee</familyName></personName></creator><creator xml:id="au8" creatorRole="author" affiliationRef="#af1"><personName><givenNames>Jing</givenNames><familyName>Tao</familyName></personName></creator><creator xml:id="au9" creatorRole="author" affiliationRef="#af4"><personName><givenNames>Yang</givenNames><familyName>Zhao</familyName></personName></creator></creators><affiliationGroup><affiliation xml:id="af1" countryCode="US" type="organization"><unparsedAffiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</unparsedAffiliation></affiliation><affiliation xml:id="af2" countryCode="US" type="organization"><unparsedAffiliation>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</unparsedAffiliation></affiliation><affiliation xml:id="af3" countryCode="US" type="organization"><unparsedAffiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</unparsedAffiliation></affiliation><affiliation xml:id="af4" countryCode="US" type="organization"><unparsedAffiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</unparsedAffiliation></affiliation></affiliationGroup><keywordGroup xml:lang="en" type="author"><keyword xml:id="kwd1">scientific workflows</keyword><keyword xml:id="kwd2">Grid workflows</keyword><keyword xml:id="kwd3">scientific data management</keyword><keyword xml:id="kwd4">problem‐solving environments</keyword><keyword xml:id="kwd5">dataflow networks</keyword></keywordGroup><fundingInfo><fundingAgency>NSF/ITR</fundingAgency><fundingNumber>0225676 (SEEK)</fundingNumber><fundingNumber>CCR‐00225610 (Chess)</fundingNumber><fundingNumber>0225673 (GEON)</fundingNumber><fundingNumber>0325963 (ROADNet)</fundingNumber></fundingInfo><fundingInfo><fundingAgency>DOE SciDAC</fundingAgency><fundingNumber>DE‐FC02‐01ER25486 (SDM)</fundingNumber></fundingInfo><fundingInfo><fundingAgency>NIH/NCRR</fundingAgency><fundingNumber>1R24 RR019701‐01</fundingNumber></fundingInfo><fundingInfo><fundingAgency>Biomedical Informatics Research Network Coordinating Center (BIRN‐CC)</fundingAgency></fundingInfo><fundingInfo><fundingAgency>NSF/DBI</fundingAgency><fundingNumber>0078296 (Resurgence)</fundingNumber></fundingInfo><abstractGroup><abstract type="main" xml:lang="en"><title type="main">Abstract</title><p>Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call <i>scientific workflows</i>. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.</p></abstract></abstractGroup></contentMeta></header></component></istex:document></istex:metadataXml><mods version="3.6"><titleInfo lang="en"><title>Scientific workflow management and the Kepler system</title></titleInfo><titleInfo type="abbreviated" lang="en"><title>SCIENTIFIC WORKFLOW MANAGEMENT AND THE KEPLER SYSTEM</title></titleInfo><titleInfo type="alternative" contentType="CDATA" lang="en"><title>Scientific workflow management and the Kepler system</title></titleInfo><name type="personal"><namePart type="given">Bertram</namePart><namePart type="family">Ludäscher</namePart><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation><affiliation>Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</affiliation><description>Correspondence: Department of Computer Science and Genome Center, UC Davis, Davis, CA 95616, U.S.A.</description><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Ilkay</namePart><namePart type="family">Altintas</namePart><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Chad</namePart><namePart type="family">Berkley</namePart><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Dan</namePart><namePart type="family">Higgins</namePart><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Efrat</namePart><namePart type="family">Jaeger</namePart><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Matthew</namePart><namePart type="family">Jones</namePart><affiliation>National Center for Ecological Analysis and Synthesis, UC Santa Barbara, Santa Barbara, CA 93101, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Edward A.</namePart><namePart type="family">Lee</namePart><affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Jing</namePart><namePart type="family">Tao</namePart><affiliation>San Diego Supercomputer Center, UC San Diego, San Diego, CA 92093, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><name type="personal"><namePart type="given">Yang</namePart><namePart type="family">Zhao</namePart><affiliation>Department of Electrical Engineering and Computer Sciences, UC Berkeley, Berkeley, CA 94720, U.S.A.</affiliation><role><roleTerm type="text">author</roleTerm></role></name><typeOfResource>text</typeOfResource><genre type="article" displayLabel="article"></genre><originInfo><publisher>John Wiley & Sons, Ltd.</publisher><place><placeTerm type="text">Chichester, UK</placeTerm></place><dateIssued encoding="w3cdtf">2006-08-25</dateIssued><dateCaptured encoding="w3cdtf">2004-06-01</dateCaptured><dateValid encoding="w3cdtf">2005-04-27</dateValid><copyrightDate encoding="w3cdtf">2006</copyrightDate></originInfo><language><languageTerm type="code" authority="rfc3066">en</languageTerm><languageTerm type="code" authority="iso639-2b">eng</languageTerm></language><physicalDescription><internetMediaType>text/html</internetMediaType><extent unit="figures">9</extent><extent unit="references">71</extent></physicalDescription><abstract lang="en">Many scientific disciplines are now data and information driven, and new scientific knowledge is often gained by scientists putting together data analysis and knowledge discovery ‘pipelines’. A related trend is that more and more scientific communities realize the benefits of sharing their data and computational services, and are thus contributing to a distributed data and computational community infrastructure (a.k.a. ‘the Grid’). However, this infrastructure is only a means to an end and ideally scientists should not be too concerned with its existence. The goal is for scientists to focus on development and use of what we call scientific workflows. These are networks of analytical steps that may involve, e.g., database access and querying steps, data analysis and mining steps, and many other steps including computationally intensive jobs on high‐performance cluster computers. In this paper we describe characteristics of and requirements for scientific workflows as identified in a number of our application projects. We then elaborate on Kepler, a particular scientific workflow system, currently under development across a number of scientific data management projects. We describe some key features of Kepler and its underlying Ptolemy II system, planned extensions, and areas of future research. Kepler is a community‐driven, open source project, and we always welcome related projects and new contributors to join. Copyright © 2005 John Wiley & Sons, Ltd.</abstract><note type="funding">NSF/ITR - No. 0225676 (SEEK); No. CCR‐00225610 (Chess); No. 0225673 (GEON); No. 0325963 (ROADNet); </note><note type="funding">DOE SciDAC - No. DE‐FC02‐01ER25486 (SDM); </note><note type="funding">NIH/NCRR - No. 1R24 RR019701‐01; </note><note type="funding">Biomedical Informatics Research Network Coordinating Center (BIRN‐CC)</note><note type="funding">NSF/DBI - No. 0078296 (Resurgence); </note><subject lang="en"><genre>keywords</genre><topic>scientific workflows</topic><topic>Grid workflows</topic><topic>scientific data management</topic><topic>problem‐solving environments</topic><topic>dataflow networks</topic></subject><relatedItem type="host"><titleInfo><title>Concurrency and Computation: Practice and Experience</title></titleInfo><titleInfo type="abbreviated"><title>Concurrency Computat.: Pract. Exper.</title></titleInfo><name type="personal"><namePart type="given">Geoffrey C.</namePart><namePart type="family">Fox</namePart></name><name type="personal"><namePart type="given">Dennis</namePart><namePart type="family">Gannon</namePart></name><genre type="journal">journal</genre><subject><genre>article-category</genre><topic>Research Article</topic></subject><identifier type="ISSN">1532-0626</identifier><identifier type="eISSN">1532-0634</identifier><identifier type="DOI">10.1002/(ISSN)1532-0634</identifier><identifier type="PublisherID">CPE</identifier><part><date>2006</date><detail type="title"><title>Workflow in Grid Systems</title></detail><detail type="volume"><caption>vol.</caption><number>18</number></detail><detail type="issue"><caption>no.</caption><number>10</number></detail><extent unit="pages"><start>1039</start><end>1065</end><total>27</total></extent></part></relatedItem><relatedItem type="preceding"><titleInfo><title>Concurrency: Practice and Experience</title></titleInfo><identifier type="ISSN">1040-3108</identifier><identifier type="ISSN">1096-9128</identifier><part><date point="end">2000</date><detail type="volume"><caption>last vol.</caption><number>12</number></detail><detail type="issue"><caption>last no.</caption><number>15</number></detail></part></relatedItem><identifier type="istex">30E3B595D35F4B9A8B4605A53FA65B4A123D59F9</identifier><identifier type="DOI">10.1002/cpe.994</identifier><identifier type="ArticleID">CPE994</identifier><accessCondition type="use and reproduction" contentType="copyright">Copyright © 2005 John Wiley & Sons, Ltd.</accessCondition><recordInfo><recordContentSource>WILEY</recordContentSource><recordOrigin>John Wiley & Sons, Ltd.</recordOrigin></recordInfo></mods></metadata><enrichments><json:item><type>multicat</type><uri>https://api.istex.fr/document/30E3B595D35F4B9A8B4605A53FA65B4A123D59F9/enrichments/multicat</uri></json:item></enrichments><serie></serie></istex></record>Pour manipuler ce document sous Unix (Dilib)
EXPLOR_STEP=$WICRI_ROOT/Ticri/CIDE/explor/CyberinfraV1/Data/Istex/Corpus
HfdSelect -h $EXPLOR_STEP/biblio.hfd -nk 000526 | SxmlIndent | more
Ou
HfdSelect -h $EXPLOR_AREA/Data/Istex/Corpus/biblio.hfd -nk 000526 | SxmlIndent | more
Pour mettre un lien sur cette page dans le réseau Wicri
{{Explor lien
|wiki= Ticri/CIDE
|area= CyberinfraV1
|flux= Istex
|étape= Corpus
|type= RBID
|clé= ISTEX:30E3B595D35F4B9A8B4605A53FA65B4A123D59F9
|texte= Scientific workflow management and the Kepler system
}}
| This area was generated with Dilib version V0.6.25. |  |