Luigi

Luigi is a Python package designed for building complex pipelines of batch jobs. It manages dependency resolution, workflow orchestration, and fault tolerance within these pipelines. The framework focuses on defining task dependencies and executing batch processes. Luigi provides a simpler, more lightweight approach to task management and execution in Python. It is a mature framework for handling batch job workflows, offering tools for building, running, and configuring pipelines.