data_juicer.analysis.measure module

class data_juicer.analysis.measure.Measure

基类：object

Base class for Measure distribution.

name = 'base'

measure(*args, **kwargs)

class data_juicer.analysis.measure.KLDivMeasure

基类：Measure

Measure Kullback-Leibler divergence.

name = 'kl_divergence'

measure(p, q)

class data_juicer.analysis.measure.JSDivMeasure

基类：Measure

Measure Jensen-Shannon divergence.

name = 'js_divergence'

measure(p, q)

class data_juicer.analysis.measure.CrossEntropyMeasure

基类：Measure

Measure Cross-Entropy.

name = 'cross_entropy'

measure(p, q)

class data_juicer.analysis.measure.EntropyMeasure

基类：Measure

Measure Entropy.

name = 'entropy'

measure(p)

class data_juicer.analysis.measure.RelatedTTestMeasure

基类：Measure

Measure T-Test for two related distributions on their histogram of the same bins.

Ref: https://en.wikipedia.org/wiki/Student%27s_t-test

For continuous features or distributions, the input could be dataset stats list. For discrete features or distributions, the input could be the tags or the categories list.

name = 't-test'

static stats_to_hist(p, q)

static category_to_hist(p, q)

measure(p, q)

参数:

• p -- the first feature or distribution. (stats/tags/categories)

• q -- the second feature or distribution. (stats/tags/categories)

返回:

the T-Test results object -- ([ref](https://docs.scipy.org/doc/scipy/reference/generated/scipy.stats._result_classes.TtestResult.html#scipy.stats._result_classes.TtestResult)) # noqa: E501