data_juicer.format.tsv_formatter module#

class data_juicer.format.tsv_formatter.TsvFormatter(dataset_path, suffixes=None, **kwargs)[源代码]#

基类:LocalFormatter

The class is used to load and format tsv-type files.

Default suffixes is ['.tsv']

SUFFIXES = ['.tsv']#
__init__(dataset_path, suffixes=None, **kwargs)[源代码]#

Initialization method.

参数:
  • dataset_path -- a dataset file or a dataset directory

  • suffixes -- files with specified suffixes to be processed

  • kwargs -- extra args, e.g. delimiter = ','