tesk-core

The taskmaster and the filer of TESK, an implementation of the GA4GH Task Execution Service (TES) API for Kubernetes, brought to you by ELIXIR On-Cloud. For each task that the TESK API accepts, the taskmaster runs as a Kubernetes job of its own: it creates the task's volume, has the filer stage the inputs (if any), runs the executors one after another, has the filer stage the outputs (if any), and cleans up. The filer downloads the inputs onto the task's volume and uploads the outputs, over HTTP(S), FTP, S3 or a shared volume. Both are commands of the Python package teskcore, published as the images elixircloud/tesk-core-taskmaster and elixircloud/tesk-core-filer, and deployed with the Helm chart of TESK.

Read the documentation (the "tesk-core" section of the TESK documentation) for the taskmaster, the filer and development. Its sources are in docs/.

Quick start

The taskmaster needs a Kubernetes cluster and the TESK API: deploy TESK with its Helm chart, which sets both images. The filer also runs on its own. With Docker installed, download a file into the current directory, as the filer does for a task's input:

docker run --rm --user "$(id -u)" --volume "${PWD}:/data" \
  elixircloud/tesk-core-filer:dev inputs \
  '{"inputs": [{"url": "https://raw.githubusercontent.com/elixir-cloud-aai/tesk-core/dev/LICENSE", "path": "/data/LICENSE", "type": "FILE"}]}'

Development

The package needs Python 3.10 or later. With uv, install it with the locked development dependencies and run the tests:

uv sync --extra dev
uv run pytest

See Development in the documentation for the images, lint, CI and the docs build.

Citation

If you use tesk-core, please cite it with the metadata in CITATION.cff: on GitHub, "Cite this repository" in the sidebar of the repository formats it as APA or BibTeX.

Contributing

Contributions are welcome! See the organisation's contributing guide, and mind our code of conduct.