data_juicer.core.executor.dag_execution_mixin module#

DAG Execution Mixin for Data-Juicer Executors

This mixin provides DAG execution planning and monitoring that can be integrated into existing executors to provide intelligent pipeline analysis and execution tracking.

class data_juicer.core.executor.dag_execution_mixin.DAGExecutionMixin[源代码]#

基类：object

Mixin that provides DAG-based execution planning and monitoring.

This mixin can be integrated into any executor to provide: - DAG execution planning - Execution monitoring tied to DAG nodes - Event logging with DAG context

__init__()[源代码]#

Initialize the DAG execution mixin.

get_dag_execution_status() → Dict[str, Any][源代码]#

Get DAG execution status.

visualize_dag_execution_plan() → str[源代码]#

Get visualization of the DAG execution plan.

get_dag_execution_plan_path() → str[源代码]#

Get the path to the saved DAG execution plan.

reconstruct_dag_state_from_events(job_id: str) → Dict[str, Any] | None[源代码]#

Reconstruct DAG execution state from event logs.

This method has been decomposed into smaller, focused methods for better maintainability and testability.

参数:

job_id -- The job ID to analyze

返回:

Dictionary containing reconstructed DAG state and resumption information

resume_dag_execution(job_id: str, dataset, ops: List) → bool[源代码]#

Resume DAG execution from the last known state.

参数:

  • job_id -- The job ID to resume

  • dataset -- The dataset to process

  • ops -- List of operations to execute

返回:

True if resumption was successful, False otherwise