Notebooks Tutorial

Welcome to the Notebooks Tutorial Library of Data Juicer Hub! This branch contains a series of Jupyter notebooks to help you get started with Data Juicer quickly.

🚀 Quick Start

In addition to the methods below, you can also try JupyterLab Playground with Tutorials.

Method 1: Github Codespace

1. Launch Codespace

   • Click the Code button in this repository and select the Codespaces tab

   • Click Create codespace on notebook (the + icon) to start the environment

   • Wait a moment and you’ll see the VSCode Web interface

2. Select and Run Notebooks

   • Find the notebooks folder in the left file directory

   • Click on the notebook file you’re interested in

   • Click the kernel selector in the top right corner and choose the data-juicer-hub environment (located in the .venv directory)

   • Start running!

Method 2: Google Colab

Click the links below to run the tutorials online in Google Colab:

Chapter	Title	Colab Link
01	Getting Started
02	Building Recipes
03	Data Formats and Loading
04	DJ Dataset API
05	Operators Usage
06	Analysis and Visualization
07	Distributed Processing with Ray
08	Preprocessing
09	Multimodal Data Processing
10	Advanced Dataset Configuration

📝 Tutorial Content Overview

• 01_Getting_Started - Data Juicer core concepts and quick start guide

• 02_Building_Recipes - Design and construction of data processing recipes

• 03_Data_Formats_and_Loading - Comprehensive guide to data formats and loading methods

• 04_DJ_Dataset_API - Complete Dataset API guide

• 05_Operators_Usage - Detailed explanation of data processing operators (both YAML and Python modes)

• 06_Analysis_and_Visualization - Data analysis and visualization tools

• 07_Distributed_Processing_with_Ray - Ray distributed processing framework integration

• 08_Preprocessing - Data preprocessing scripts

• 09_Multimodal_Data_Processing - Multimodal data processing capabilities

• 10_Advanced_Dataset_Configuration - Advanced Dataset configuration options

💡 Recommended Learning Paths

Quick Start

01 → 02 → 05 → 06 → 07

For: Learners who want to quickly understand and use Data Juicer’s core capabilities

Data Integration

03 → 08, 09, 10

For: Developers who already understand Data Juicer’s core capabilities and need to integrate their own data into processing pipelines

Programming and Customization

01 → 03 → 04 → 05 → 07

For: Engineers who want to flexibly customize data processing pipelines with code

📚 More Resources

• Data Juicer Official Documentation

• Data Juicer Hub

Happy Learning! 🎉