Skip to content

Tutorial

DJ-Cookbook
Installation Guide
Quick Start

docs

Operator Schemas 算子提要
Dataset Configuration Guide
“Bad” Data Exhibition
DJ-SORA
DataJuicer Agents
DJ_service
How-to Guide for Developers
Distributed Data Processing in Data-Juicer
Sandbox
Awesome Data-Model Co-Development of MLLMs

demos

Demos

hub

Data Recipe Gallery
Refine Alpaca-CoT Config Files
Notification System
BLOOM Config Files
Redpajama Config Files

tools

Distributed Fuzzy Deduplication Tools
Auto Evaluation Toolkit
GPT EVAL: Evaluate your model with OpenAI API
Evaluation Results Recorder
Format Conversion Tools
Multimodal Tools
Post Tuning Tools
Label Studio Service Utility
Metrics for video generation
VBench metrics
Postprocess tools
Preprocess Tools

thirdparty

LLM Ecosystems
Third-party Model Library

API Reference

API Reference

en|v1.4.4

Language

English 简体中文

Version

main v1.4.4 v1.4.3 v1.4.2 v1.4.1 v1.4.0

data_juicer.ops¶

data_juicer.ops.filter

data_juicer.core

Copyright © 2024, Data-Juicer Team

Made with Sphinx and @pradyunsg's Furo