Pipelines

frogbox.pipelines.pipeline

Pipeline

Pipeline(
    config,
    models,
    losses,
    trainer,
    evaluator,
    checkpoint=None,
    checkpoint_keys=None,
)

Bases: ABC

Pipeline abstract base class.

is_local_main_process property

is_local_main_process

True for one process per server.

is_main_process property

is_main_process

True for one process only.

run_name property

run_name

Get name of current run.

gather_for_metrics

gather_for_metrics(input_data, use_gather_object=False)

Gathers input_data and potentially drops duplicates in the last batch if on a distributed system. Should be used for gathering the inputs and targets for metric calculation.

Wrapper around Accelerator.gather_for_metrics.

install_callback

install_callback(
    event,
    callback,
    engine="trainer",
    only_main_process=False,
    **kwargs
)

Install callback in pipeline.

Parameters:

• event

  (MatchableEvent) –

  Event to trigger callback.

• callback

  (callable) –

  Callback function. Should take a single argument pipeline and return nothing.

• engine

  (str, default: 'trainer' ) –

  Which engine to install callback in. Defaults to "trainer".

• only_main_process

  (bool, default: False ) –

  Install only in main process. Only affects distributed setups.

• kwargs

  (keyword arguments, default: {} ) –

  Optional keyword arguments to be passed to callback.

log

log(data)

Log data to tracker(s).

print

print(*args, **kwargs)

Drop in replacement of print() to only print once per server.

run

run()

Run pipeline.

frogbox.pipelines.supervised

SupervisedPipeline

SupervisedPipeline(
    config, checkpoint=None, checkpoint_keys=None
)

Bases: Pipeline

Supervised pipeline.

Parameters:

• config

  (SupervisedConfig) –

  Pipeline configuration.

• checkpoint

  (path - like, default: None ) –

  Path to experiment checkpoint.

• checkpoint_keys

  (list of str, default: None ) –

  List of keys for objects to load from checkpoint. Defaults to all keys.