Notification System

Data Juicer supports notifications for operators that inherit from the NotificationMixin class. Each operator defines its own notification configuration, which can include email, Slack, and DingTalk notifications.

Basic Configuration

To enable notifications for an operator:

- human_preference_annotation_mapper:
    # Enable notifications for this operator
    notification_config:
      enabled: true  # Master switch for notifications
      
      # Email channel configuration
      email:
        enabled: true  # Enable email notifications
        smtp_server: "smtp.example.com"
        smtp_port: 587
        username: "your-email@example.com"
        sender_email: "your-email@example.com"
        recipients: ["recipient1@example.com", "recipient2@example.com"]
        subject: "Notification from Data Juicer"
      
      # Slack channel configuration (optional)
      slack:
        enabled: false  # Disable Slack notifications

Channel-Specific Settings

Each notification channel (email, Slack, DingTalk) can be independently enabled or disabled:

notification_config:
  enabled: true  # Master switch for notifications
  
  email:
    enabled: true  # Enable email notifications
    # Email settings...
  
  slack:
    enabled: false  # Disable Slack notifications
    # Slack settings (won't be used since disabled)
  
  dingtalk:
    enabled: true  # Enable DingTalk notifications
    # DingTalk settings...

Email Configuration

Email notifications support various server configurations:

Standard SMTP with STARTTLS:

email:
  enabled: true
  smtp_server: "smtp.example.com"
  smtp_port: 587
  username: "your.username@example.com"
  sender_email: "your.username@example.com"
  sender_name: "Your Name"  # Optional
  recipients: ["recipient1@example.com", "recipient2@example.com"]

Direct SSL Connection (e.g., Gmail):

email:
  enabled: true
  smtp_server: "smtp.gmail.com"
  smtp_port: 465
  use_ssl: true
  username: "your.username@gmail.com"
  sender_email: "your.username@gmail.com"
  recipients: ["recipient1@example.com"]

Alibaba Email Server:

email:
  enabled: true
  smtp_server: "smtp.alibaba-inc.com"
  smtp_port: 465
  use_ssl: true
  include_port_in_address: true  # Include port in server address
  use_mime_multipart: false      # Use simple MIMEText
  username: "your.username@alibaba-inc.com"
  sender_email: "your.username@alibaba-inc.com"
  recipient_separator: ";"       # Use semicolons for recipients
  add_message_id: true           # Add Message-ID header
  recipients: ["recipient1@example.com", "recipient2@example.com"]

Secure Password Handling

For security, passwords should be provided via environment variables:

# General email password
export DATA_JUICER_EMAIL_PASSWORD="your_password"

# Server-specific passwords (preferred)
export DATA_JUICER_SMTP_GMAIL_COM_PASSWORD="your_gmail_password"
export DATA_JUICER_SMTP_ALIBABA_INC_COM_PASSWORD="your_alibaba_password"

# Slack webhook
export DATA_JUICER_SLACK_WEBHOOK="your_slack_webhook_url"

Certificate Authentication

For enhanced security, you can use TLS client certificates:

email:
  enabled: true
  use_cert_auth: true
  client_cert_file: "/path/to/client.crt"
  client_key_file: "/path/to/client.key"
  smtp_server: "smtp.example.com"
  smtp_port: 587
  sender_email: "notifications@example.com"
  recipients: ["recipient@example.com"]

Examples

See these example configuration files: