Chariots ML

Module that handles all of the Machine learning integration.

This modeule provides helpers for the most popular ML frameworks (sci-kit learn and keras for now) as well as the BaseMlOp class to allow you to create Ops for non supported frameworks (or custom algorithms)

class chariots.ml.MLMode[source]

Bases: enum.Enum

mode in which to put the op (prediction of training) enum

FIT = 'fit'
FIT_PREDICT = 'fit_predict'
PREDICT = 'predict'
class chariots.ml.BaseMLOp(mode: chariots.ml._ml_mode.MLMode, op_callbacks: Optional[List[chariots.pipelines.callbacks._op_callback.OpCallBack]] = None)[source]

Bases: chariots.pipelines.ops._loadable_op.LoadableOp

an BaseMLOp are ops designed specifically to be machine learning models (whether for training or inference). You can initialize the op in three distinctive ml mode:

  • FIT for training the model

  • PREDICT to perform inference

  • FIT_PREDICT to do both (train and predict on the same dataset

the usual workflow is to a have a training and a prediction pipeline. and to:

  • execute the training pipeline:

  • save the training pipeline

  • reload the prediction pipeline

  • use the prediction pipeline

here is an example:

first create your pipelines:

>>> train = Pipeline([
...     Node(IrisFullDataSet(), output_nodes=["x", "y"]),
...     Node(PCAOp(MLMode.FIT_PREDICT), input_nodes=["x"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.FIT), input_nodes=["x_transformed", "y"])
... ], 'train')

>>> pred = Pipeline([
...     Node(PCAOp(MLMode.PREDICT), input_nodes=["__pipeline_input__"], output_nodes="x_transformed"),
...     Node(LogisticOp(MLMode.PREDICT), input_nodes=["x_transformed"], output_nodes=['pred']),
...     Node(FromArray(), input_nodes=['pred'], output_nodes='__pipeline_output__')
... ], 'pred')

and then to train your pipelines and make some predictions:

>>> response = client.call_pipeline(train)
>>> client.save_pipeline(train)
>>> client.load_pipeline(pred)
>>> response = client.call_pipeline(pred, [[1, 2, 3, 4]])
>>> response.value
[1]

If you want to create a new MLOp class (to accommodate an unsupported framework for instance), you need to define:

  • how to fit your op with the fit method

  • how to perform inference with your op with the predict method

  • define how to initialize a new model with the _init_model method

and eventually you can change the serializer_cls class attribute to change the serialization format of your model

Parameters

op_callbacks – OpCallbacks objects to change the behavior of the op by executing some action before or after the op’s execution

__init__(mode: chariots.ml._ml_mode.MLMode, op_callbacks: Optional[List[chariots.pipelines.callbacks._op_callback.OpCallBack]] = None)[source]
Parameters

mode – the mode to use when instantiating the op

property allow_version_change

whether or not this op accepts to be loaded with the wrong version. this is usually False but is useful when loading an op for retraining

execute(*args, **kwargs)[source]

executes the model action that is required (train, test or both depending in what the op was initialized with

abstract fit(*args, **kwargs)[source]

fits the inner model of the op on data (in args and kwargs) this method must not return any data (use the FIT_PREDICT mode to predict on the same data the op was trained on)

load(serialized_object: bytes)[source]

Receives serialize bytes of a newer version of this class and sets the internals of he op accordingly.

Parameters

serialized_object – the serialized bytes of this op (as where outputed by the serialize method

property mode

the mode this op was instantiated with

property op_version

the version the op uses to pass to the pipeline to identify itself. This differs from the __version__ method in that it can add some information besides the class Fields (for instance last training time for ML Ops)

abstract predict(*args, **kwargs) → Any[source]

the method used to do predictions/inference once the model has been fitted/loaded

serialize() → bytes[source]

serializes the object into bytes (to be persisted with a Saver) to be reloaded in the future (you must ensure the compatibility with the load method

Returns

the serialized bytes representing this operation

serializer_cls

alias of chariots.ml.serializers._dill_serializer.DillSerializer

training_update_version = 'patch'