data_juicer.ops.mapper.punctuation_normalization_mapper module#

class data_juicer.ops.mapper.punctuation_normalization_mapper.PunctuationNormalizationMapper(*args, **kwargs)[source]#

Bases: Mapper

Mapper to normalize unicode punctuations to English punctuations in text samples.

__init__(*args, **kwargs)[source]#

Initialization method.

Parameters:

  • args – extra args

  • kwargs – extra args

process_batched(samples)[source]#