doot.workflow._interface

Tasks are the main abstractions managed by Doot

JOBS create tasks

TASKS have actions

ACTIONS are individual atomic steps of a task, given the detailed information necessary to perform the step.

Jobs, as they can control refication order, can add setup and teardown tasks. This can allow interleaving, or grouping.

Communication paths: Job -> Task : by creation Task -> Action : by creation Action -> Task : by return value, updating task state dict Task -> Job : by reference to the job

Task -> Task = Postboxes Action -> Action = Action -> Task State -> Action

Type Aliases

ActionReturn

Enums

`QueueMeta_e`	available ways a task can be activated for running
`RelationMeta_e`	What types+synonyms of task relation there can be,
`TaskMeta_e`	Flags describing properties of a task,
`TaskStatus_e`	Enumeration of the different states a task/artifact can be in.
`ArtifactStatus_e`	States an artifact can be in
`ActionResponse_e`	Description of how a Action went.

Protocols

`Status_ep`	Base class for protocol classes.
`ActionSpec_i`	For things that need building, but don't have a separate factory
`InjectSpec_i`	For things that need building, but don't have a separate factory
`RelationSpec_i`	Base class for protocol classes.
`TaskSpec_i`	The data spec of a task. is created from TOML data
`Action_p`	holds individual action information and state, and executes it
`Artifact_i`	Something which describes a file system location,
`TaskName_p`	The Main protocol describing a Strang.
`TaskFactory_p`	Base class for protocol classes.
`SubTaskFactory_p`	Base class for protocol classes.
`Task_p`	Base class for protocol classes.
`Job_p`	builds tasks
`Task_i`	Meta information for a task

Classes

DelayedSpec

Module Contents

doot.workflow._interface.ActionReturn: TypeAlias = Maybe[dict | bool | ActionResponse_e]

class doot.workflow._interface.QueueMeta_e(*args, **kwds)[source]

Bases: enum.Enum

available ways a task can be activated for running onRegister/auto : activates automatically when added to the task network reactive : activates if an adjacent node completes

default : activates only if uses queues the task, or its a dependencyOf

default

onRegister

reactive

reactiveFail

auto

class doot.workflow._interface.RelationMeta_e(*args, **kwds)[source]

Bases: enum.Enum

What types+synonyms of task relation there can be, in the form Obj {rel} Y,

eg: cake dependsOn baking. or: baking requirementFor cake. or: eatingCake conflictsWith givingCake

needs

blocks

classmethod default() → RelationMeta_e[source]

Return type:: RelationMeta_e

class doot.workflow._interface.TaskMeta_e[source]

Bases: enum.StrEnum

Flags describing properties of a task, stored in the Task_p instance itself.

TASK

JOB

TRANSFORMER

INTERNAL

JOB_HEAD

CONCRETE

DISABLED

EPHEMERAL

IDEMPOTENT

REQ_TEARDOWN

REQ_SETUP

IS_TEARDOWN

IS_SETUP

THREAD_SAFE

STATEFUL

STATELESS

VERSIONED

classmethod default() → jgdv.Maybe[source]

Return type:: jgdv.Maybe

class doot.workflow._interface.TaskStatus_e(*args, **kwds)[source]

Bases: enum.Enum

Enumeration of the different states a task/artifact can be in. The state is stored in the task object itself.

Before a task object hsa been created, the tracker provides the status according to what specs exist for the task name.

NAMED

DECLARED

DEFINED

DISABLED

INIT

WAIT

READY

RUNNING

SKIPPED

HALTED

FAILED

SUCCESS

TEARDOWN

DEAD

classmethod default() → TaskStatus_e[source]

Return type:: TaskStatus_e

classmethod pre_set() → set[source]

Return type:: set

classmethod success_set() → set[source]

Return type:: set

classmethod fail_set() → set[source]

Return type:: set

class doot.workflow._interface.ArtifactStatus_e(*args, **kwds)[source]

Bases: enum.Enum

States an artifact can be in

DECLARED

STALE

TOCLEAN

EXISTS

class doot.workflow._interface.ActionResponse_e(*args, **kwds)[source]

Bases: enum.Enum

Description of how a Action went.

SUCCEED

FAIL

SKIP

SKIP_GROUP

SKIP_TASK

SUCCESS

class doot.workflow._interface.Status_ep[source]

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...

classmethod default() → Any[source]

Return type:: Any

classmethod pre_set() → set[source]

Return type:: set

classmethod success_set() → set[source]

Return type:: set

classmethod fail_set() → set[source]

Return type:: set

class doot.workflow._interface.ActionSpec_i[source]

Bases: jgdv._abstract.protocols.general.Buildable_p, Protocol

For things that need building, but don’t have a separate factory TODO add type parameter

do: jgdv.Maybe[jgdv.structs.strang.CodeReference]

args: list[Any]

kwargs: jgdv.structs.chainguard.ChainGuard

fun: jgdv.Maybe[jgdv.Func]

class doot.workflow._interface.InjectSpec_i[source]

Bases: jgdv._abstract.protocols.general.Buildable_p, Protocol

For things that need building, but don’t have a separate factory TODO add type parameter

from_spec: dict

from_state: dict

from_target: dict

literal: dict

with_suffix: jgdv.Maybe[str]

apply_from_spec( parent: dict | TaskSpec_i | Task_p, ) → dict[source]

Parameters:: parent (dict | TaskSpec_i | Task_p)
Return type:: dict

apply_from_state(parent: dict | Task_p) → dict[source]

Parameters:: parent (dict | Task_p)
Return type:: dict

apply_literal(val: Any) → dict[source]

Parameters:: val (Any)
Return type:: dict

validate( control: Task_p | TaskSpec_i, target: Task_p | TaskSpec_i, *, only_spec: bool = False, ) → bool[source]

Parameters:

control (Task_p | TaskSpec_i)
target (Task_p | TaskSpec_i)
only_spec (bool)

Return type:

bool

validate_details( control: Task_p | TaskSpec_i, target: Task_p | TaskSpec_i, *, only_spec: bool = False, ) → dict[source]

Parameters:

control (Task_p | TaskSpec_i)
target (Task_p | TaskSpec_i)
only_spec (bool)

Return type:

dict

class doot.workflow._interface.RelationSpec_i[source]

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...

Marks: ClassVar[type[enum.Enum]]

target: TaskName_p | Artifact_i

relation: RelationMeta_e

object: jgdv.Maybe[TaskName_p | Artifact_i]

constraints: dict[str, str]

inject: jgdv.Maybe[InjectSpec_i]

to_ordered_pair( obj: RelationTarget, *, target: jgdv.Maybe[TaskName_p] = None, ) → tuple[jgdv.Maybe[RelationTarget], jgdv.Maybe[RelationTarget]][source]

Parameters:

obj (RelationTarget)
target (jgdv.Maybe[TaskName_p])

Return type:

tuple[jgdv.Maybe[RelationTarget], jgdv.Maybe[RelationTarget]]

instantiate( *, obj: jgdv.Maybe[RelationTarget] = None, target: jgdv.Maybe[RelationTarget] = None, ) → RelationSpec_i[source]

Parameters:

obj (jgdv.Maybe[RelationTarget])
target (jgdv.Maybe[RelationTarget])

Return type:

RelationSpec_i

forward_dir_p() → bool[source]

Return type:: bool

accepts( control: Task_i | TaskSpec_i, target: Task_i | TaskSpec_i, ) → bool[source]

Parameters:

control (Task_i | TaskSpec_i)
target (Task_i | TaskSpec_i)

Return type:

bool

class doot.workflow._interface.TaskSpec_i[source]

Bases: Protocol

The data spec of a task. is created from TOML data

_default_ctor: ClassVar[str]

_blocking_groups: ClassVar[tuple[str, Ellipsis]]

Marks: ClassVar[enum.Enum]

name: TaskName_p

doc: jgdv.Maybe[list[str]]

sources: list[jgdv.Maybe[TaskName_p | pathlib.Path]]

actions: list[ActionSpec_i]

required_for: list[ActionSpec_i | RelationSpec_i]

depends_on: list[ActionSpec_i | RelationSpec_i]

setup: list[ActionSpec_i | RelationSpec_i]

cleanup: list[ActionSpec_i | RelationSpec_i]

on_fail: list[ActionSpec_i | RelationSpec_i]

version: str

priority: int

ctor: jgdv.structs.strang.CodeReference

queue_behaviour: QueueMeta_e

meta: set[TaskMeta_e]

property extra: jgdv.structs.chainguard.ChainGuard[source]

Return type:: jgdv.structs.chainguard.ChainGuard

param_specs() → list[source]

Return type:: list

class doot.workflow._interface.Action_p[source]

Bases: Protocol

holds individual action information and state, and executes it

class doot.workflow._interface.Artifact_i[source]

Bases: jgdv.structs.locator._interface.Location_p, Protocol

Something which describes a file system location, with a possible identifier, and metadata

priority: int

get_status() → ArtifactStatus_e[source]

Return type:: ArtifactStatus_e

reify( other: pathlib.Path | jgdv.structs.locator._interface.Location_p, ) → jgdv.Maybe[Artifact_i][source]

Parameters:: other (pathlib.Path | jgdv.structs.locator._interface.Location_p)
Return type:: jgdv.Maybe[Artifact_i]

class doot.workflow._interface.TaskName_p[source]

Bases: jgdv.structs.strang._interface.Strang_p, Protocol

The Main protocol describing a Strang.

with_head() → Self[source]

Return type:: Self

is_head() → bool[source]

Return type:: bool

with_cleanup() → Self[source]

Return type:: Self

is_cleanup() → bool[source]

Return type:: bool

pop_generated() → Self[source]

Return type:: Self

class doot.workflow._interface.TaskFactory_p( *, spec_ctor: jgdv.Maybe[type] = None, task_ctor: jgdv.Maybe[type] = None, job_ctor: jgdv.Maybe[type] = None, )[source]

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...

Parameters:

spec_ctor (jgdv.Maybe[type])
task_ctor (jgdv.Maybe[type])
job_ctor (jgdv.Maybe[type])

build( data: jgdv.structs.chainguard.ChainGuard | dict | TaskName_p | str, ) → TaskSpec_i[source]

Parameters:: data (jgdv.structs.chainguard.ChainGuard | dict | TaskName_p | str)
Return type:: TaskSpec_i

instantiate( obj: TaskSpec_i, *, extra: jgdv.Maybe[collections.abc.Mapping | bool] = None, ) → TaskSpec_i[source]

Parameters:

obj (TaskSpec_i)
extra (jgdv.Maybe[collections.abc.Mapping | bool])

Return type:

TaskSpec_i

merge( *, bot: dict | TaskSpec_i, top: dict | TaskSpec_i, suffix: jgdv.Maybe[str | Literal[False]] = None, ) → TaskSpec_i[source]

Parameters:

bot (dict | TaskSpec_i)
top (dict | TaskSpec_i)
suffix (jgdv.Maybe[str | Literal[False]])

Return type:

TaskSpec_i

make( obj: TaskSpec_i, *, ensure: jgdv.Maybe = None, inject: jgdv.Maybe[tuple[InjectSpec_i, Task_i]] = None, parent: jgdv.Maybe[Task_i] = None, ) → Task_i[source]

Parameters:

obj (TaskSpec_i)
ensure (jgdv.Maybe)
inject (jgdv.Maybe[tuple[InjectSpec_i, Task_i]])
parent (jgdv.Maybe[Task_i])

Return type:

Task_i

action_group_elements( obj: TaskSpec_i, ) → collections.abc.Iterable[ActionSpec_i | RelationSpec_i][source]

Parameters:: obj (TaskSpec_i)
Return type:: collections.abc.Iterable[ActionSpec_i | RelationSpec_i]

class doot.workflow._interface.SubTaskFactory_p[source]

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...

generate_names( obj: TaskSpec_i, ) → list[TaskName_p][source]

Parameters:: obj (TaskSpec_i)
Return type:: list[TaskName_p]

generate_specs( obj: TaskSpec_i | Artifact_i | DelayedSpec, ) → list[dict][source]

Parameters:: obj (TaskSpec_i | Artifact_i | DelayedSpec)
Return type:: list[dict]

class doot.workflow._interface.Task_p(spec: TaskSpec_i)[source]

Bases: Protocol

Base class for protocol classes.

Protocol classes are defined as:

class Proto(Protocol):
    def meth(self) -> int:
        ...

Such classes are primarily used with static type checkers that recognize structural subtyping (static duck-typing).

For example:

class C:
    def meth(self) -> int:
        return 0

def func(x: Proto) -> int:
    return x.meth()

func(C())  # Passes static type check

See PEP 544 for details. Protocol classes decorated with @typing.runtime_checkable act as simple-minded runtime protocols that check only the presence of given attributes, ignoring their type signatures. Protocol classes can be generic, they are defined as:

class GenProto[T](Protocol):
    def meth(self) -> T:
        ...

Parameters:: spec (TaskSpec_i)

property name: TaskName_p[source]

Return type:: TaskName_p

property spec: TaskSpec_i[source]

Return type:: TaskSpec_i

property status: TaskStatus_e[source]

Return type:: TaskStatus_e

property priority: int[source]

Return type:: int

property internal_state: dict[source]

Return type:: dict

log( msg: str, level: int = logmod.DEBUG, prefix: jgdv.Maybe[str] = None, ) → None[source]

Parameters:

msg (str)
level (int)
prefix (jgdv.Maybe[str])

Return type:

None

prepare_actions() → None[source]

Return type:: None

get_action_group( group_name: str, ) → list[ActionSpec_i][source]

Parameters:: group_name (str)
Return type:: list[ActionSpec_i]

class doot.workflow._interface.Job_p(spec: TaskSpec_i)[source]

Bases: Task_p, Protocol

builds tasks

Parameters:: spec (TaskSpec_i)

expand_job() → list[source]

Return type:: list

class doot.workflow._interface.Task_i(spec: TaskSpec_i)[source]

Bases: Task_p, Protocol

Meta information for a task

Parameters:: spec (TaskSpec_i)

_default_flags: ClassVar[set[TaskMeta_e]]

_version: str

_help: tuple[str, Ellipsis]

doc: tuple[str, Ellipsis]

class doot.workflow._interface.DelayedSpec(**kwargs: Any)[source]

Parameters:: kwargs (Any)

base: TaskName_p

target: TaskName_p

inject: list[InjectSpec_i]

applied: dict

overrides: dict