chariots.savers

Savers are used to persist and reload ops. A saver can be viewed as the basic abstraction of a file system (interprets path) and always has a root path (that represents the path after which the saver will start persisting data).

To use a specific saver, you need to pass it as a parameter of your OpStoreServer so that the op_store_client in terms

knows how and where to persist your ops

>>> my_saver = FileSaver(app_path)
>>> op_store_client = OpStoreServer(my_saver, db_url)

For now chariots only provides a basic FileSaver and a GoogleStorageSaver but there are plans to add more in future releases (in particular to support more cloud service providers such as aws s3).

savers are used to persist and retrieve information about ops, nodes and pipeline (such as versions, persisted versions, datasets, and so on).

to create your own saver, you can subclass the BaseSaver class

class chariots.op_store.savers.FileSaver(root_path: str)

Bases: chariots.op_store.savers._base_saver.BaseSaver

a saver that persists to the local file system of the machine the Chariots saver is running on.

load(path: str) → bytes

loads the bytes serialized at a specific path

Parameters

path – the path to load the bytes from.You should not include the root_path of the saver in this path: loading to /foo/bar.txt on a saver with /my/root/path as root path will load /my/root/path/foo/bar.txt

Returns

saved bytes

Raises

FileNotFoundError – if the file does not exist

save(serialized_object: bytes, path: str) → bool

saves bytes to a specific path.

Parameters

• serialized_object – the bytes to persist

• path – the path to save the bytes to. You should not include the root_path of the saver in this path: saving to /foo/bar.txt on a saver with /my/root/path as root path will create/update /my/root/path/foo/bar.txt

Returns

whether or not the object was correctly serialized.

class chariots.op_store.savers.GoogleStorageSaver(root_path: str, bucket_name: str, client_kwargs: Optional[Mapping] = None)

Bases: chariots.op_store.savers._base_saver.BaseSaver

saver to persist data mdoels and more to the google storage service.

Parameters

• root_path – the root path of where to save the data inside the bucket

• bucket – a google.could.storage Bucket object to save the data to

__init__(root_path: str, bucket_name: str, client_kwargs: Optional[Mapping] = None)

load(path: str) → bytes

loads the bytes serialized at a specific path

Parameters

path – the path to load the bytes from.You should not include the root_path of the saver in this path: loading to /foo/bar.txt on a saver with /my/root/path as root path will load /my/root/path/foo/bar.txt

Returns

saved bytes

Raises

FileNotFoundError – if the file does not exist

save(serialized_object: bytes, path: str) → bool

saves bytes to a specific path.

Parameters

• serialized_object – the bytes to persist

• path – the path to save the bytes to. You should not include the root_path of the saver in this path: saving to /foo/bar.txt on a saver with /my/root/path as root path will create/update /my/root/path/foo/bar.txt

Returns

whether or not the object was correctly serialized.

class chariots.op_store.savers.BaseSaver(root_path: str)

Bases: abc.ABC

abstraction of a file system used to persist/load assets and ops this can be used on the actual local file system of the machine the Chariots server is running or on a bottomless storage service (not implemented, PR welcome)

To create a new Saver class you only need to define the Save and Load behaviors

Parameters

root_path – the root path to use when mounting the saver (for instance the base path to use in the the file system when using the FileSaver)

__init__(root_path: str)

Initialize self. See help(type(self)) for accurate signature.

load(path: str) → bytes

loads the bytes serialized at a specific path

Parameters

path – the path to load the bytes from.You should not include the root_path of the saver in this path: loading to /foo/bar.txt on a saver with /my/root/path as root path will load /my/root/path/foo/bar.txt

Returns

saved bytes

Raises

FileNotFoundError – if the file does not exist

save(serialized_object: bytes, path: str) → bool

saves bytes to a specific path.

Parameters

• serialized_object – the bytes to persist

• path – the path to save the bytes to. You should not include the root_path of the saver in this path: saving to /foo/bar.txt on a saver with /my/root/path as root path will create/update /my/root/path/foo/bar.txt

Returns

whether or not the object was correctly serialized.