Task Execution Service (TES)

Provides a standard mechanism for orchestrating complex analyses across different compute environments

Analysing data across the world can make genomics research more powerful by connecting many large-scale datasets for simultaneous analysis. Such investigations use complex methods, like aligning multiple sequences to the human reference genome to identify potentially pathogenic variants. These analyses often involve up to hundreds of thousands of computational tasks, which can take considerable time and compute power to execute. Developed by the GA4GH Cloud Work Stream, the Task Execution Service (TES) API provides a standard mechanism for orchestrating these complex analyses across different compute environments.

Jump to...

Publications Announcements Version history Documents Implementations Contributors News

Benefits

Connects complex workflow engines to new compute systems, saving time and resources
Enables federated analysis
Can be implemented in both cloud environments and high performance computing systems

Target users

Researchers

Image summary: TES helps researchers plug into new compute environments to run analysis tasks.

USE THIS PRODUCT GET INVOLVED

THEME

Cloud genomics

Publications

Community resources

Dive deeper into this product! To support large scale federated analysis, institutions and organisations employ queuing systems that send tasks out to high performance computers (HPCs) or cloud environments. Every compute system, however, is unique — each vendor uses incompatible APIs for running batch tasks. Because of these discrepancies, researchers carrying out federated analysis must employ unique code for each. TES provides a standard mechanism for orchestrating these complex analyses across different compute environments. On the back end, the TES API wraps around an institution’s HPC system or cloud environment, and then manages the deployment, scheduling, running, and cleanup of tasks while providing status updates and logging information back to the researcher.

Announcements