Data-Juicer Sphinx Documentation Template#

This is a unified documentation build template designed for the Data-Juicer ecosystem. Built on Sphinx and pydata-sphinx-theme, it provides multi-version and multi-language documentation capabilities, ensuring consistent documentation appearance and user experience across all subprojects.

Features#

Unified Appearance: All subprojects share the same documentation theme and styling.
Multi-Version Support: Automatically builds documentation for multiple Git branches and tags.
Multi-Language Support: Supports both English and Chinese by default.
Ecosystem Interconnectivity: Enables seamless navigation between different project documentations via header external links.
Markdown-Friendly: Automatically discovers and integrates Markdown documents within the project.

Project Structure#

data-juicer-sphinx/
├── docs/
│   └── sphinx_doc/                              # Sphinx documentation build directory
│       ├── build_versions.py                    # Multi-version build script (main entry point)
│       ├── make.bat / Makefile                  # Build scripts
│       ├── redirect.html                        # Redirect page
│       └── source/                              # Documentation source files
│           ├── conf.py                          # Sphinx configuration file
│           ├── custom_myst.py                   # Custom MyST extension
│           ├── external_links.yaml              # External project link configuration
│           ├── index.rst / index_ZH.rst               # Home page (customization recommended)
│           ├── docs_index.rst / docs_index_ZH.rst     # Documentation index page (customization recommended)
│           ├── api.rst                          # API documentation index (customization recommended)
│           ├── _static/                         # Static assets
│           │   ├── custom.css                   # Custom styles
│           │   └── images/                      # Logos and icons
│           └── _templates/                      # Custom templates
│               └── version-language-switcher.html
├── guides/                                      # Usage guides
├── pyproject.toml                               # Project configuration
├── README.md
└── README_ZH.md

Quick Start#

Build the simplest English Data-Juicer Sphinx documentation (without API docs):

git clone https://github.com/datajuicer/data-juicer-sphinx.git

uv pip install .

cd docs/sphinx_doc
export PROJECT="data-juicer-sphinx"
python build_versions.py -A -l en

Documentation#

Here

Core Principles#

Isolated Build Environment (Git Worktree)#

Creates an independent Git worktree for each version (branch/tag) at .worktrees/<version>.
Automatically cleans up after building (unless KEEP_WORKTREES=True is set in docs/sphinx_doc/build_versions.py) to avoid polluting the main working directory.

Documentation Content Aggregation#

Automatically scans the entire worktree to collect all .md and .rst files (excluding directories like outputs, sphinx_doc, .github, etc.).
Copies these files into a unified Sphinx source directory: docs/sphinx_doc/source/.
(Customized for Data-Juicer operator documentation) For subdirectories under operators/, automatically generates corresponding index.rst and index_ZH.rst files to facilitate categorized operator indexing.

Frequently Asked Questions#

Q1: Build fails with “module not found” error#

A: Ensure all dependencies are installed before building:

uv pip install .

Q2: API documentation isn’t generated#

A: Check the following:

Ensure you didn’t use the --no-api-doc or -A flags
Verify your project contains importable Python modules
Confirm the CODE_ROOT environment variable is correctly set

Q3: External links aren’t displayed#

Verify that external_links.yaml is configured correctly
Ensure the PROJECT environment variable is properly set
Check the browser console for JavaScript errors

Q4: Chinese documentation links don’t exist#

A: Ensure:

Chinese documentation files end with _ZH.md or _ZH.rst
index_ZH.rst exists and is correctly configured

Q5: Page doesn’t exist after switching versions#

A: Documentation structures may differ between versions:

Older versions might lack certain new pages
Version switching attempts to access the same path; if unavailable, it redirects to the homepage

Contribution Guide#

Contributions and improvements to this template are warmly welcomed! ❤

DOCS
- Enable This Template for Your Project
- Test Document