March 19-22, 2019, Livermore, CA
You are hereby invited to participate in the DOE workflow workshop and hackathon to be held March 19-22, 2019 in Livermore, CA. This event is intended to bring together people in the larger DOE community who are developing and using workflow technology. Days 1 and 2 will be the workshop followed by a 2-day workflow hackathon, where people will have the opportunity to do hands-on work with the various tools/capabilities used around the DOE. Details for both events are provided below.
Materials from the workshop will be posted on this site for archival purposes.
Registration is required, as is badging for this event.
Please find the workshop and hackathon calls for participation below. If you plan to join us, we request that you register by sending an email to the org committee at:
indicate the following:
- Which days you plan to attend the workshop (March 19 & 20).
- Which days you plan to join us for one or more hackathon sessions (March 21 & 22).
- Affiliation (e.g., ANL, ORNL, ...)
- There is no registration fee for the workshop and hackathon
- However, attendees are requested to bring $25/day (cash) of attendance for food and drinks, which will consist of:
- Coffee in the morning
- Coffee and snacks in afternoon break
- Any dietary restrictions should be reported directly to Letty
- Badging is required for site access
- Badging info can be found here
- Badging should be done well in advance, particularly for non-US citizens
Registering for the workshop will subscribe you to an email list for news and announcements. Alternatively, anyone may request to be added to the email@example.com list as follows:
Workflow Email ListWe have created a mailing list that will persist after this event for purposes of sharing workflow-related info amoungst interested parties. To subscribe:
- Send an email to: DOE-WORKFLOW-WORKSHOP-NEWS-request@LISTSERV.LLNL.GOV
- With the following command in the body of the email:
- subscribe doe-workflow-workshop-news <firstname> <lastname>
Organizing Committee (firstname.lastname@example.org)
- Jay Billings, ORNL
- Julien Capul, CEA
- Robert Clay, SNL
- Dan Laney, LLNL
- Dave Montoya, LANL
- Letty Quihuis, email@example.com, +1-925-294-4950
Special NIF Tour (CANCELED DUE TO SHOT MONDAY NOON)
We regret to inform those who had signed up for the NIF tour that the tour was canceled due to a "high yield" shot which is now scheduled for noon Monday.
By popular demand from a small group, we are providing a means to Skype into the workshop event. We will try to get the Skype online by 8:30AM both days, so you can bring up your system in advance to work out the kinks if you're not a regular Skype user.
Trouble Joining? Try Skype Web App
Join by Phone
Conference ID: 6206911
DOE Workflow Workshop – Tues-Weds March 19-20, 2019, Livermore, CA
Date: Tues-Weds, March 19-20, 2019, 9:00-17:00 (PST)
Location: Bldg 904 Auditorium, Sandia National Labs, Livermore, CA
Call for Participation
The goal of this workshop is to bring together users of workflows and those developing workflow tools in the broader DOE community to update one another on the state of practice, present emerging technologies and tools, and to enable new collaborations in the workflow area. All topics related to workflow are welcome, including but not limited to:
- Workflow management systems
- Data Management [simulation and experimental]
- Application of workflow systems to HPC use-cases
- Resource management and scheduling
- Containers and containerized computing as they apply to workflow systems
If you are interested in presenting at the workshop send a title, abstract, and bio to firstname.lastname@example.org. Speakers will be notified of acceptance on or before February 25th. Talks will be limited to 30 minutes, including Q&A.
In addition to the scheduled talks, we will provide time on day 2 for 5-minute lightening talks, timeslots for which will be first-come-first-serve on sign up sheet provided day 1.
Tuesday March 19, 2019
Opening Session (09:00-10:30)
- 09:00 Welcome and Introduction
- 09:15 Keynote: G.E. Orient - Workflow Platform Foundational to Authoring, Communicating and Executing Credible Computational Models
- 10:00 S. Jha - Toward Common Components for Open Workflow Systems: The RADICAL-Cybertools Experience
Workflow Systems (11:00-12:30)
- 11:00 E. Friedman-Hill - The Sandia Analysis Workbench Next Generation Workflow System
- 11:30 J.M. Wozniak - Swift/T: An HPC-oriented Workflow Tool
- 12:00 K. Chard - Parsl: Pervasive Parallel Programming in Python
Lunch (12:30-14:00) [GROUP PHOTO OUT FRONT OF BLDG 904]
Workflow Systems and In-transit Analysis (14:00-15:00)
- 14:00 J. Koning - Merlin Workflow: Machine Learning Driven Ensembles
- 14:30 J. Capul - Executing HPC workflows in-transit with PaDaWAn and Redis
Mini Break (15:00-15:10)
Workflow Tools (15:10-16:10)
- 15:10 P.P. Pébaÿ - The Automatic Report Generator (ARG)
- 15:40 T. Dahlgren - Sina: A Gateway to Modern Simulation Data Management
Panel: Day 1 Speakers (16:10-16:50)
Day 1 Wrap Up (16:50-17:00)
Day 1 Adjourned (17:00)
No-host Dinner at Uncle Yu's in downtown Livermore (18:00-?)
Wednesday March 20, 2019
Opening Session (09:00-10:30)
- 09:00 Welcome and Introduction
- 09:15 Keynote: H. Krishnan - Streaming Realtime Workflows at the Light Sources
- 10:00 C. Snavely - Complex Workflows at Scale: the NERSC Superfacility Project
Workflow Container Solutions (11:00-12:30)
- 11:00 D. Montoya - Opening Remarks
- 11:10 S. Canon - Containers at NERSC: Shifter for HPC and Spin for Services
- 11:30 T. Stirm - ACE's Genesis
- 11:50 J. Kincl - Hosting Workflow Systems on Kubernetes at OLCF
- 12:10 P. Grubel - Containerized Application Workflows using the Build and Execution Environment (BEE)
Workflow Management Systems (14:00-15:30)
- 14:00 T. Patki - Flux: Using Next-Generation Resource Management and Scheduling Infrastructure for Exascale Workflows
- 14:30 M. Salim - Balsam: Simplifying Large-scale Science Campaigns on a Supercomputer
- 15:00 Lightening Talks - sign up sheet out front
Panel: Day 2 Speakers (16:00-16:50)
Day 2 Wrap Up (16:50-17:00)
Workshop Adjourned (17:00)
DOE Workflow Hackathon – Thurs-Fri March 21-22, 2019, Livermore, CA
Dates:Thurs-Friday, March 21-22, 2019 (PST), 9:00-17:00 (PST)
Location: Sandia National Labs, Livermore, CA (rooms to be announced later)
Call for Participation
Persons interested in organizing a session are invited to send a title and abstract describing their proposed session(s) to email@example.com. Session organizers will be expected to build the necessary software stacks and prepare materials as needed in advance of the hackathon. An open-network Linux server will be provided, or session organizers are welcome to host their own. Hackathon participants will be provided network access, and are expected to bring their own laptops with Unix-type OS (i.e., Linux or Mac OS) and tested applications they would like to integrate into workflows. Participants must send an email to firstname.lastname@example.org request accounts on the open server (accounts on the open server will be limited to US citizens only). Session organizers will be given early access to ensure smooth operation of their software stack.
Description and Goals
- Bring together people actively developing and/or using workflow tools in the community.
- Provide an opportunity for application developers and users to interact with authors and experts of workflow tools.
- Provide an opportunity to get hands-on experience with the current workflow tools/capabilities guided by expert developers and users.
- Thurs March 21, 2019, two 3-hour sessions (9:00-12:00, 13:30 – 16:30) (PST)
- Friday March 22, 2019, two 3-hour sessions (9:00-12:00, 13:30 – 16:30) (PST)
- Thurs & Friday, Daily wrap up & open discussion, 16:30-17:00
- Friday 17:00 Event Adjuourned
Proposed Hackathon Topics
- LLNL - data management tools + Cassandra
- CEA - in-situ workflow infrastructure (PaDaWAn)
- SNL - next-gen workflow (NGW) (getting started wtith NGW)
- SNL - credibility workflow (NGW) (building and running complex workflows)
- NGA - automatic report generation (ARG)
- LANL/NERSC - Containers in workflow, including BEE / Charliecloud and Shifter
- ANL - Swift/T
- LANL - Expanding GitLab CI to support workflow
The final schedule for hackathon sessions will be determined based on community interest and response. All topics listed above will be accomodated if there is sufficient interest for the session.