chariots.runners

runners are used to execute Pipelines: they define in what order and how each node of the pipeline should be executed.

For the moment Chariots only provides a basic sequential runner that executes each operation of a pipeline one after the other in a single threat however we have plans to introduce new runners (process and thread based ones as well as some cluster computing one) in future releases.

You can use runners directly if you want to execute your pipeline manually:

>>> runner = SequentialRunner()
>>> runner.run(is_odd_pipeline, 5)
True

or you can set the default runner of your app and it will be used every time a pipeline execution is called:

>>> my_app = PipelinesServer(app_pipelines=[is_odd_pipeline], runner=SequentialRunner(),
...                          op_store_client=op_store_client,
...                          import_name="my_app")

class chariots.pipelines.runners.SequentialRunner

Bases: chariots.pipelines.runners._base_runner.BaseRunner

runner that executes every node in a pipeline sequentially in a single thread.

run(pipeline: chariots.pipelines._pipeline.Pipeline, pipeline_input: Optional[Any] = None)

runs a pipeline, provides it with the correct input and extracts the results if any

Parameters

• pipeline – the pipeline to run

• pipeline_input – the input to be given to the pipeline

Returns

the output of the graph called on the input if applicable

class chariots.pipelines.runners.BaseRunner

Bases: abc.ABC

a runner is used to define the execution behavior of a Pipeline. there main entry point is the run method

>>> runner.run(is_odd_pipeline, 3)
True

To create a new runner (for instance to execute your pipeline on a cluster) you only have to override run method and use the Pipeline’s class methods (for instance you might want to look at extract_results, execute_node)

abstract run(pipeline: chariots.pipelines.Pipeline, pipeline_input: Optional[Any] = None)

runs a pipeline, provides it with the correct input and extracts the results if any

Parameters

• pipeline – the pipeline to run

• pipeline_input – the input to be given to the pipeline

Returns

the output of the graph called on the input if applicable