Doc API

Sandbox Hub Agents

English 简体中文

main v1.5.5 v1.5.4 v1.5.3 v1.5.2 v1.5.1 v1.5.0 v1.4.6 v1.4.5 v1.4.4 v1.4.3 v1.4.2 v1.4.1 v1.4.0

Home

Tutorial

DJ-Cookbook
Installation Guide
Quick Start

docs

Operator Schemas 算子提要
Dataset Configuration Guide
“Bad” Data Exhibition
Cache Management
DJ-SORA
DJ_service
How-to Guide for Developers
Distributed Data Processing in Data-Juicer
Dataset Export
Job Management
Partitioned Processing with Checkpointing
Data Tracing
Awesome Data-Model Co-Development of MLLMs

operators

Aggregator
Deduplicator
Filter
Mapper
Formatter
Grouper
Selector
Pipeline
- llm_ray_vllm_engine_pipeline
- vlm_ray_vllm_engine_pipeline

demos

Demos
Note for dataset path

tools

Distributed Fuzzy Deduplication Tools
Auto Evaluation Toolkit
GPT EVAL: Evaluate your model with OpenAI API
Evaluation Results Recorder
Format Conversion Tools
Multimodal Tools
Post Tuning Tools
Label Studio Service Utility
Metrics for video generation
VBench metrics
Postprocess tools
Preprocess Tools

thirdparty

LLM Ecosystems
Third-party Model Library

Related projects

Sandbox Hub Agents

Pipeline¶

llm_ray_vllm_engine_pipeline
vlm_ray_vllm_engine_pipeline

On this page

Pipeline

Previous topk_specified_field_selector Next llm_ray_vllm_engine_pipeline

© 2024, Data-Juicer Team — Built with Sphinx & Data-Juicer Theme

↵ to select ↑↓ to navigate esc to close