tslumen.scheduling module

Provides options for the scheduling of tasks and computation.

class tslumen.scheduling.Scheduler(config: Optional[Union[tslumen.scheduling.Scheduler.Config, dict]] = None)

Bases: object

Wrapper around joblib’s Parallel + delayed to integrate with Hydra’s config and offer some syntactic sugar to the execution.

class Config(n_jobs: Optional[int] = - 2, prefer: Optional[str] = 'processes', verbose: int = 0, timeout: Optional[float] = None, backend: Optional[str] = None, pre_dispatch: Any = '2 * n_jobs', batch_size: Any = 'auto', temp_folder: Optional[str] = None, max_nbytes: Optional[Any] = '1M', mmap_mode: Optional[str] = 'r', require: Optional[str] = None, progress_disable: bool = False)

Bases: object

A dataclass representing joblib’s Parallel default parameters.

A dataclass representing joblib’s Parallel default parameters.

backend: Optional[str] = None

batch_size: Any = 'auto'

max_nbytes: Optional[Any] = '1M'

mmap_mode: Optional[str] = 'r'

n_jobs: Optional[int] = -2

pre_dispatch: Any = '2 * n_jobs'

prefer: Optional[str] = 'processes'

progress_disable: bool = False

require: Optional[str] = None

temp_folder: Optional[str] = None

timeout: Optional[float] = None

verbose: int = 0

run(fn: Callable, args: Sequence[tuple], desc: str = '') → list

Runs a single function with multiple args

Parameters

• fn (Callable) – Function to be executed in parallel.

• args (Sequence[tuple]) – A sequence of sets of arguments to pass on to fn.

• desc (str) – A description to accompany the progress bar.

Returns

A list with the return values of each function.

Return type

list

class tslumen.scheduling.TqdmParallel(*args: Any, progress_disable: bool = True, **kwargs: Any)

Bases: joblib.parallel.Parallel

For using tqdm with joblib’s Parallel

Parameters

depth (int, optional) – The depth of objects printed.

print_progress() → None

Display the process of the parallel execution only a fraction of time, controlled by self.verbose.