data_juicer.core.ray_analyzer module#

class data_juicer.core.ray_analyzer.RayAnalyzer(cfg: Namespace | None = None)[source]#

Bases: object

Analyzer that uses Ray for distributed stats computation.

Computes filter stats in parallel via Ray, then uses Ray’s native aggregation (Mean, Std, Min, Max, Count) for overall analysis. No data is materialized to pandas — all computation stays distributed.

__init__(cfg: Namespace | None = None)[source]#
run(dataset=None, load_data_np: Annotated[int, Gt(gt=0)] | None = None, skip_export: bool = False, skip_return: bool = False)[source]#